The Tyranny of Data
Are you tired of taking surveys yet? Of having your opinion taken about restaurants, or car salesmen, or your internet shopping experience, of your reading habits, or of anything else?
For that matter, are you tired of having opinions taken about your work—as a sales person, or a teacher, or a deliverer of some or other product or service?
Well, prepare to get exhausted. We are living in the age of data. As increases in processor speeds and storage have made computer analysis cheap, easy and pervasive, companies, governments, medical providers and others have begin to ask a vast array of questions about what they do and how they do it—questions for which answers can be crunched quickly and easily.
The current craze is to call all this opinion gathering “assessment,” but it’s really opinion solicitation in order to figure out whether you liked the way you were treated in some interaction, or if you were satisfied with the good or service you received. The surveys are then used to assess the success (or failure) the service or good provider had in interacting with a client/receiver. They are also used to determine how best to get potential clients/users to keep using a given product, or to expand the item’s market reach.
At its best, this constant opinion soliciting leads to meaningful improvements in both goods and services over time. To the degree that better (and worse) ways of identifying, serving and satisfying users can be determined through opinion data, changes can be instituted that reflect “best practices.” Hypothetically, knowing what works and what doesn’t makes it possible to constantly get better—the path to progress.
There are, however, at least two significantly negative consequences of this assessment frenzy.
The first is the obsessive focus on that which is being measured. That is, since thing X is being measured, thing X draws most of the attention of policy-makers, reformers, journalists and others. Think No Child Left Behind: the US tests for competency in various areas; schools (and increasingly teachers) are held directly responsible for student performance on the NCLB tests; teachers teach to the test, and schools and engage in various schemes to bump their test numbers up quite regardless of whether the students have actually learned anything. The obsessive focus on THE DATA can warp behavior in profound ways.
The second problem is the flip side of the first: in focusing on the things you can—or want to—measure, it may well be that you miss the things that really matter.
To continue with schools as an example, it turns out that one of the best predictors of a student’s likelihood of doing things like graduating high school or graduating college is—surprise surprise!—whether or not the student’s parents (or caregivers) graduated from high school or college, etc. The children of better off, better educated parents (the two variables interact to a substantial degree in the United States) are, in general, more successful in school than are those whose parents are less educated and less well off. In turn, given the class stratification of America’s schools, the children of better off, better educated parents go to “better” schools, on average, than do those from poorer, less formally educated backgrounds.
In other words, it’s economic class, rather than school or teacher performance, that seems to best account for student success in the classroom.
But we can’t assess that—or don’t want to—in a way that reinforces the American mythology of the self-made aristocracy of talent.
So instead we have collected mountains of data—and soon will have mountains more—that tell us the best ways to do things that have, at best, marginal influence on the outcomes we claim to seek—e.g., holding schools and teachers directly responsible for student performance. Moreover, current US policy of slashing support for higher ed, which leads both to higher tuition and reduced financial aid and grant opportunities for the neediest students, will only reinforce the advantages held by the best off of our citizens even as “objective” test data will prove that schools that serve poor people are failing.
It’s the tyranny of data. And it’s only going to get worse.
