The Limits Of Data In Education

Plus, Understanding The Difference Between Curriculum and Learning Standards

In the years leading up to his death, my mentor Clay Christensen battled many ailments. He wondered often about when things were all said and done, as a deeply religious, man, how would he be viewed when he had his final interview with God?

Clay’s ultimate conclusion (which I wrote about here) revolved around how he viewed data, of which he always harbored suspicions (for those reading—stay with me! I promise there’s a relevant hook for education).

First, Clay was fond of pointing out that data are created by humans. Data aren’t some objective things that magically exist in the world. Humans must create categories, observe phenomena, and make decisions about what the “data” are. There’s subjectivity in data, no matter how we might wish otherwise, as data are based on constructs that humans have created at one point or another.

Second, data, as he was fond of saying, are, by definition, backward looking. They are unable, by definition, to tell you anything about the future in and of themselves. And by the time data become available, the phenomenon has already passed. Even for those that talk about “real-time” data, they are, by definition, somewhat of an oxymoron. What’s more, data are only convincingly and clearly available about the distant past after people have had time to sort through them and scrub them.

In my latest piece for Forbes, I shared Clay’s conclusions about why humans needed data, but God didn’t.

I then shared why those thoughts—independent of one’s religion or whether you believe in God—hold significant implications for how we shape outcomes-based data in higher education. They point to a need to figure out how to capture individual learners’ desires for progress in metrics. My concern is that the majority of outcomes-based data we use today in higher education are fundamentally supply-side in nature—that is, they view the world from the perspective of those institutions that deliver education.

The problem is that using data in that case might not match the demand-side of the equation. What is success for an individual learner in their specific circumstances? What happens when what success looks like for a particular individual doesn’t match up with what is right for the institution?

How can we cease judging institutions on supply-side metrics that capture the average and instead evaluate them based on how they facilitate the individual progress each learner desires in their lives? How do we capture learner voice in outcomes-based metrics?

You can read the full piece, “The Flaw Behind Today’s Outcomes-Based Data In Higher Education,” here, some of which draws on the research Bob Moesta and I did for our book Choosing College. My ultimate takeaway is we need a lot more work done in this area, so after reading the piece and hearing my early thoughts, I’d love to hear your takes of who is doing good work in this area to solve this mismatch.

What’s Going On With the Workforce?

Speaking of data, on Future U Ben Casselman of the New York Times and Rachel Carlson, CEO of Guild Education, joined me and Jeff Selingo to talk about why there are so many “Help Wanted” signs right now and how higher education might fill the void for giving workers needed skills. Ben observed that one challenge in explaining the current economy is that so much of the data available don’t paint a clear picture of what’s taking place.

For my part, this is precisely why relying on good theory is so important to help guide urgent decisions. You can hear on the podcast how Guild Education is tackling this with its employers and academic partners and what that might portend. Of note—Rachel also observes in the podcast that Guild is seeing some of the same challenges in the supply-side vs demand-side data that I discussed in my Forbes article. Check out the theories each hold—and what questions they are asking—given the limits around the workforce data in the episode here.

Standards And Curriculum Aren’t The Same

In the latest edition of Class Disrupted, I dug into Diane Tavenner’s observation in the previous shows that she was in favor of common learning standards (say, the Common Core) but against a common curriculum. Diane geeked out on what is curriculum—and how it differs from what standards are and why that matters in our heated conversations around what gets taught. For those who were inflamed around the Common Core debate, the episode is worth a listen.

Three Video Interviews

I highly recommend checking out the following conversations:

1)    Tom Vander Ark, founder of Getting Smart and former head of the Gates Foundation, joined me to talk about what he’s seeing on the ground in K–12 schools in the wake of the pandemic—including the bright spots, but also where he sees confusion and struggles.

2)    Mark Van Ryzin of the University of Oregon spoke with me about the benefits of small-group instruction, both academically and socially.

3)    I joined The Innovation Show with Aidan McCullen to revisit my 2008 book, Disrupting Class, over two sessions. The in-depth conversations surfaced many themes that weren’t always discussed when the book first came out, and we delved into many of the lessons from innovation for educators and organizations more generally. You can watch them here and here. It was fun to revisit my first book with Clay Christensen and Curtis Johnson so many years later.

MacKenzie Scott Gift To Global Citizen Year

MacKenzie Scott has become known in higher education circles for her large charitable gifts to “overlooked” colleges, including a great many Historically Black Colleges and Universities. Well, she’s no longer just supporting colleges and universities.

Last week, Global Citizen Year, which offers a leadership program for high school graduates before they go to college, announced that it had received $12 million from Scott to turbocharge its vision.

When I was writing Choosing College, I became convinced that significantly more students should be taking a “gap year”—or what Abby Falik, CEO of Global Citizen Year, calls a “purpose year” (you can hear her talk about that on my YouTube channel)—to build in them a clearer sense for why they are enrolling in college and what they hope to get out of the experience. This would greatly boost the success of many individuals who struggle today in college, as they enroll just because it’s what someone else expected them to do.

Will this gift from Scott be what the country needs to help make a purpose year before college more mainstream? I certainly hope so.

Thanks as always for reading, writing, and listening.