What Are We Measuring? The EdTech Edition

Image of Wordle built from the U.S. National Educational Technology Plan via drapestakes.blogspot

Education, educational equity, and education technology. I’ve always wanted to be involved in the first, I’m now working for a nonprofit dedicated to the second, and my incredibly talented, intelligent, and entrepreneurial friend James has gotten me totally hooked on the third. Now I see EdTech everywhere I go, and I am loving it! But of course, my number one question whenever I am learning about new EdTech is: where is the Social Impact Assessment?

Easy to ask, not easy to answer. EdSurge for FastCompany put out a great article recently that led me to discover a cool new tool from the NewSchools Venture Fund that begins the daunting but important task of mapping the edtech world. The tool is good looking and easy to use. Here’s the overview shot of the map:

And when you drill down to data – which is of course the section I want to showcase – you see something like this:

So far, so good. But once we get down to Student Information Systems (SIS), Reporting Systems, and Data Warehousing, we get back to the hard questions. I actually love how interactive and tech-driven almost all of the mapped companies’ services appear (I don’t have any demos, so I’m only talking about what information I get from the websites). But exactly what data are we gathering? The basics all appear to be there – grades, reading comprehension, math skills, attendance…easily quantifiable metrics that are cornerstones to assessing the educational experience. Look how pretty this all got wrapped up by the winner of GOOD’s “redesign the report card” competition. But what I can’t necessarily tell from the map and the individual websites is whether any of these systems are gathering, storing, and reporting on some of the more qualitative aspects of social impact in the education sector. Are students more engaged in their classrooms? Do they collaborate more with their peers? Do they enjoy learning? Are they inspired and motivated to advance their learning beyond the classroom? Do they believe they can succeed in life? Are they compelled to teach others, formally or informally?

Working at Teach For America for just about 6 weeks now, but having had several deep conversations about the validity and actualization of our core values, I’ve thought a lot about transformational change. It is plain to me (and I believe to many others) that the things we are accustomed to measuring do not paint the whole picture, especially when it comes to transformational change. But how can we quantify inspiration and motivation? And clearly measuring a child’s motivation to learn is only one half of the challenge, because we then need to measure their follow-through on that motivation. I’m going to make a guess here (and I’ll be happily proved wrong if anyone wants to speak up) that all the student information systems and data warehousing needs end when a child leaves the school or school system in which they were enrolled. And should they switch schools, their data most likely will not follow them. So how can we really tell the long-term, transformational impact of how we are educating our kids? My first thoughts go to gamification of self-reporting data – make it a rewards-system game for kids to enter their own data to track their learning experience. All the systems I saw seemed to be completely aimed at the adult users, with little interaction and engagement from students themselves – except for one: lookred.

Check out lookred’s “what we do” section and you’ll see why they describe themselves as “a student centred experience” serving “a new breed of Digital Natives are moving up through education systems across the world.” Despite the glaringly obvious typo in this next piece of information about one of the offerings from lookred, quoted from the company website, the information is beautiful: “lookred® Spaces begins to address these changing practices and behaviours by providing a platform for education that is more in line with the consumer experience, and draws on inspiration from ‘knowing as much about our learners as Tesco/Wal-mart/Amazon do about thier customers’.  By applying logic traditionally constrained to marketing and advertising domains we can begin to not only better understand individual learners needs and preferences but to specifically target appropriate resources, content and support.” This goes right back to my last post about User Behavior Analytics (…) and how they can be leveraged for social impact assessment. Is lookred® Spaces the tool we can use to gather those missing metrics about transformational change? Will this platform be able to link up (by opt-in) to future Facebook, LinkedIn, and other pages to continue to track transformational change over time?

It’s a brave new world for data collection and for transformational change. How are we going to marry these concepts into a Social Impact Assessment that truly helps us assess the effects of our actions on our stakeholders as well as on society and the world around us? I’m really exhilarated to see so much of this work happening in the education sector. I see two main take-aways from the mapping of the edtech world: doing the hard work of determining what transformational change really is and creating new platforms for metrics and reporting is critical, and engaging students/stakeholders in that work is revolutionary.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: