Digital Record Deluge Threatens to Swamp States
MikeDotta / Shutterstock.com
States allocate less than one-hundredth of one percent of their budget on archives despite exponential data growth.
The dramatic growth in digital records produced by state agencies presents significant challenges as archivists seek to preserve the ongoing stream of information for the historical record.
There has been a staggering 1,693 percent growth in state and territorial electronic records between 2006 and 2016, according to a new report from national organizations representing state CIOs and archivists. Despite the rapid growth, states’ average spending on archive and records management is just .007 percent of their annual budgets.
“Sustained attention and resources are needed to ensure the long-term management and accessibility of our nation’s electronic records,” the report states, warning of the “challenges for states that are not prepared.”
The report by the National Association of State Chief Information Officers and Council of State Archivists, entitled “State Archiving in the Digital Era,” is fashioned as a ‘playbook’ for state officials, outlining policies and actions state leaders can take to manage the increasingly difficult task of digital preservation.
The challenges outlined in the playbook are both daunting and familiar to anyone who spends time on the internet (which, these days, includes mostly everyone).
For instance, IT officials and archivists are constantly trying to manage how government officials use social media—a seemingly never-ending source of news from local governments, to states, all the way up to the tweeter-in-chief.
The report also notes states must cope with technology “evolving at an accelerating rate,” creating situations where older file formats can no longer be easily read.
States also change private sector vendors as new companies come to the fore and others fall behind (or close up shop altogether). The report urges states to have an exit strategy for the preservation and transfer of public data when they move from one private sector contractor to another.
This is all before governments even begin to worry about losing data to malicious acts by hackers, or guarding particularly sensitive data—such as certain police records or personnel data—from unauthorized access.
Unfortunately, not all state agencies and officials understand the undertaking required to identify and preserve electronic records. According to Amy Glasscock, a senior policy analyst at NASCIO and co-author of the report, “getting it on the radar” of everyone who needs to be involved is difficult, particularly when dealing with “larger fires” that demand people’s attention day-to-day.
With a lack of attention comes limited funding.
“Budgets have not improved significantly in recent years for archives and records management,” said Michelle Gallinger, a co-author of the report who works for the Council of State Archivists. “The reason that this playbook identifies dedicating funding as the toughest play of all is because there hasn't been strong interest in budgeting for archives and records management in recent years.”
Meanwhile, the “volume and complexity of electronic government records continues to increase at an exponential rate.”
State and territories digital records have climbed to 1371.1 terabytes, according to the report. That is more than is currently held by the U.S. National Archives, which weighs in at 800 terabytes of electronic records. To put that in perspective, one terabyte is the equivalent of around two million photos, 500 hours of video, or 18.75 million four-page documents.
With state governments delving into the deployment of “smart” internet-of-things sensors, connected infrastructure, autonomous vehicles, police body cameras and other rich sources of data, the growth of publicly-owned data is not slowing down anytime soon.
“We really need to get on top of it before it’s harder to manage,” Glasscock said.
Mitch Herckis is Senior Editor and Director of Strategic Initiatives for Government Executive's Route Fifty and is based in Washington, D.C.
NEXT STORY: Most Big Cities Are Planning for Autonomous Vehicles