I spent a good chunk of the day reading A Byte at the Apple: Rethinking Education Data for the Post-NCLB Era, which Fordham put out a last year, but I missed at the time. A few thoughts.
After reading this book, you only have the barest hints of the existence of vendors in the educational data sphere. They recount at some length the failings of government bureaucracy and the inconveniences of politics, mis-aligned priorities, and annoyances like privacy concerns, but the role of commercial vendors, and the IT industry writ large in this woeful history is omitted. When projects go over budget, fail to meet user needs, and trap data in silos, it isn't always or exclusively the customers' fault. A casual read through A Byte of the Apple would suggest that many or most large education data projects are done entirely in house. How many are really? I don't know.
Given Fordham's ideological leanings, one can infer (as I believed going in) that most of this work is already done by private vendors, because otherwise Fordham would be calling for privatizing the educational data industry.
But if you are going to point out mis-aligned interests of actors, you need to include vendors, who have their own interests. Kenneth Wong misses this entirely. Nice job. It can be in a vendor's interest to limit the use of data in their systems. This is a significant part of the puzzle.
A couple authors touch on SIF, with some optimism. Of course, the further into the SIF world you get, both in terms of SIF politics and the actual technical implementation, the queasier one usually gets. I'm not saying SIF is bad, or that it doesn't work. But it is not nearly as good as it could be, in large part because of the parochial interests of key vendors, whose approach seems to be to maintain a large slice of a small pie, rather than a smaller piece of a large pie (where the size of the pie represents the number of schools using SIF). SIF occupies incredibly strategic territory in this discussion, and it seems destined to remain indefinitely in a limbo where it is not so good as to become very widely used and useful, but not so bad that anybody could generate the momentum to displace it.
The problem is that if you're going to write an 150 page book about educational data systems, with the intention of, you know, improving the field, you really ought to know enough to say, "SIF is ok, but actually not nearly good enough, and these geeks need a kick in the ass." Because they do. And if the authors of this book can't make that point, who will?
Educational data is a very technical subject, and the technical capacity of the authors of these papers is limited, and, quite frankly, circumscribed by class. These folks are too important to bother understanding how this stuff actually works, it is really just plumbing, so after a certain point, their analysis drops off a cliff. Unless you have a decent understanding of information architecture (or whatever you want to call it), you can't even perceive the cliff. But the problem with, say, the Data Quality Campaign, is that it only includes the simple, easy to understand and explain aspects of the problem. It doesn't even acknowledge the hard parts. And while on one hand you might want to start with the easy parts, fine, the hard parts will take a long time. The hard parts take as long to explain as the easy parts take to fix, and unfortunately, (almost) nobody is even trying to explain the hard parts, or acknowledge they exist, even when they write 150 page books on the subject.
I would be happy if instead of blowing pixie dust about "student data backpacks," Margaret Raymond would get in a little golf cart and ride over to the Stanford Center for Biomedical Informatics Research and write a little paper on concrete steps we can take to apply and emulate their work in education. They're really friendly.
Also, I'd note that their description of South Korea's comprehensive national open source administrative infrastructure, which I know too little about, is impressive, but of course leads to no recommendations of similar approaches here. Too pink for Fordham, I'd imagine.
Finally, Checker Finn's vision for how all this fits together in 2025 is just creepy:
All students carry PDAs (or cell phones) that communicate with tracking devices in the school, and Alexandra, a typical student, is no different. Using these devices as well as swipe-able ID cards, the activities that fill her day are entered into the school data system -- and, when warranted, flashed to teachers and parents. For example, each day the system calculates how much time Alexandra spends sitting and listening to the teacher, doing seat work, taking formative assessments using her PDA, reading independently, doing math problems at a computer, playing outside, etc. This information is used by teachers and analysts to determine how Alexandra might better use her time and the school's learning resources.
Look, this isn't abstract, because I'll have two girls in high school in 2025, and I don't want them going to that school. I don't want them living their lives that way. I don't want them to learn that constant electronic surveillance is necessary, normal or benign. Because it isn't. The authors do a pretty good job throughout the proceeding pages in arguing that privacy restrictions should be lessened, but Finn blows it for me with his vision of educational telescreens in every backpack.