Education officials are assessing and untangling all the ways schools have been reporting data and making decisions and filtering them into common metrics and a usable format.
THE BIDEN administration is set to give educators and school leaders the very thing that the previous administration refused them: a centralized data collection to help them understand the impact of the pandemic on students and teachers alongside the status of in-person learning for schools and districts across the country.
The directive, which was included in an executive order signed by the president last week and falls to the Institute of Education Sciences to facilitate, is part of the Biden administration’s sprawling plan to curb COVID-19 in the U.S. and get the country’s economy and school systems back up and running. It’s a herculean task, given the country’s 13,000 school districts have, for the most part, been going it alone for the last 10 months, operating without any substantive guidance from state or federal officials.
What that means, practically speaking, for Education Department officials tasked with the job is a top-to-bottom assessment and untangling of all the different ways schools have been collecting and reporting data and making decisions about how to operate, filtering it all into common metrics and spitting it out in a usable format to help meet Biden’s ambitious goal of getting K-8 schools open in his first 100 days.
“You have 13,000 local data systems,” says Paige Kowalski, executive vice president of the Data Quality Campaign. “And because 13,000 school districts came up with their own response plan, you have 13,000 different ways of defining what in-person or hybrid is, or on grade level, or off-track.”
The initial scramble was understandable, Kowalski says, because the country was in an emergency situation. But the Trump administration, and specifically former Education Secretary Betsy DeVos, said it wasn’t the federal government’s responsibility to establish any kind of data collection about reopening plans and coronavirus cases in schools – despite school leaders begging for it.
“There was a real missed opportunity to spend the summer getting this together so that you had guidance for states and districts to start counting things in a comparable and consistent way and then aggregating that information up to the national level so that Congress can come back and begin to solve the problem,” Kowalski says.
“And we don’t know [how to solve the problem],” she continues, “because we did not collect in a common, consistent way locally and we did not have a mechanism to push that data up and aggregate it. And because we didn’t do that, there is also no ability to disaggregate it back down to understand the disparate impacts across economic, geographic and racial and ethnic indicators.”
“The fact that we lost 10 months is huge.”
The overwhelming sense is that Education Department officials should not start from scratch. A handful of education policy organizations, groups that represent educators and superintendents and even education technology companies have been trying to build out databases tracking various metrics of the pandemic’s impact on education.
The Center on Reinventing Public Education has been tracking how schools are operating since last March. Additionally, AASA, the School Superintendents association, has been working with Emily Oster, an economics professor at Brown University, to build a database that tracks COVID-19 infection rates in school districts. And NWEA, the nonprofit provider of assessment solutions, has been trying to capture the amount of academic learning loss, while the National Education Association and the American Federation of Teachers have been tracking educator layoffs – to name just a few of the ongoing efforts.
“We and others have a start on this,” says Robin Lake, who has been overseeing the database curated by researchers at the Center for Reinventing Public Education, where she is the director. “It will be important to build on that. We can’t waste time.”
But there’s a big question about exactly what metrics need to be part of the data collection, not to mention how department officials plan to patch together the various efforts.
Lake says it would make sense if the Biden administration required states to report monthly data on all their districts’ operational statuses because that data, which is embedded with federal codes, would allow department officials to know for sure how many districts and schools are open and whether the administration is meeting its goals for reopening.
It will also be important, she says, to know what assessments and instructional strategies districts are using to understand and address academic learning loss.
The database should also include the number of adult and student COVID-19 cases as well as the various health measures districts are employing so that district leaders can learn quickly how effective those measures are, Lake says.
“You cannot have a database on reopening in the face of a pandemic without including infection rates because the decision to reopen should in large part be driven by what we know about the rates,” says Noelle Ellerson Ng, associate executive director of advocacy and policy at AASA, the School Superintendents Association.
But some school superintendents, Ellerson Ng says, have voiced concerns about a database being unintentionally weaponized at the federal level by, for example, being built into accountability metrics or creating a rubric that labels schools red, yellow or green based on their opening status.
“We don’t think that’s the Biden administration’s intent at all,” Ellerson Ng says. “But we also do understand the proclivity of the federal government to say, ‘Well look at this comprehensive set of data. We know it helps inform the reopening of schools, but perhaps it could also help us evaluate this,’ or ‘Let’s build it into this accountability metric. Superintendents have no patience for that.”
Lawmakers might assume, for example, that students in school districts that didn’t reopen for in-person learning accrued more learning loss and, therefore, might want to focus funding on those districts to make up for the academic loss. But in doing so, they might completely overlook the fact that it took an incredible amount of resources for other school districts to do the heavy lifting required to reopen, and they need additional funding to keep going.
“You could find two similarly situated districts, and one just had a different political capacity to open and both still incurred the same types of cost,” Ellerson Ng says.
Many also worry about the burden of additional reporting requirements, and whether they’ll be asked to duplicate what they may already be reporting to the state.
“They need to think through how the reporting is going to be done,” Ellerson Ng says. “It’s really hard to see a scenario where this data is reported without it being another thing at the local level. Is a federal data set going to draw from existing state databases? Or is the federal government instead going to incentivize states to create datasets with parameters of what works and what doesn’t?”
Because of the local nature of education and the number of stakeholders with their hands in the pot, the effort is bound to get political quickly, especially when it comes to defining certain metrics.
“There are a lot of politics in definitions and in numerators and denominators, because when the numbers come out the finger pointing begins and the scramble for resources begins,” Kowalski says. “The actors involved want to make sure the definitions and the numerators and denominators favor them.”
For example, if one school district has 100% of its students in hybrid learning and another district has 50% of its students in hybrid learning, you might draw a conclusion from that. But if students who are in the 100% hybrid learning district are only in school one time a week, and students in the 50% hybrid learning district are in the building three times a week, the latter is actually offering more in-person learning.
Similarly, it’s not as simple as asking who has the internet at home. The equally important question is: Does that internet have the capacity to support remote learning needs, and is it fast enough to support, for example, two children and an adult working from home?
“That’s why definitions are so important,” Kowalski says. “If we rush too much, we are going to collect data that is not consistent. It might be timely, but it won’t be consistent and, therefore, it will lack a certain quality and limit the types of decisions we can make from it and the types of insights we can draw from it.”
One question that looms large for school leaders and education policy and data experts is just how comprehensive the data collection will be – whether it will be a quick effort to get schools reopen as fast as possible or whether it will lay the groundwork for an in-depth analysis of the repercussions of the pandemic.
“When I see the words, ‘fully understand the impact of the pandemic on students and educators,'” says Kowalski, referencing the language in the executive order, “to me that says create capacity and don’t let this be a one-off. Otherwise, it’s kind of a waste.”
“A one-off data collection saying how many students have the internet is an important question to ask – maybe the most important question out there right now – but that won’t help us in four years,” she says. “And we have to think of the long game here. We will be answering questions and solving the effects of this pandemic for decades.“