We just solved the biggest problem with search! Introducing NOMAD™ AI Purely Visual Search. Click the button to the right to tour NOMADTM or schedule a demo today.
The commercial use of hard disk drives began way back in 1957. Since this time, digital storage has grown exponentially in both capacity and use. Countless millions now rely on their personal information being kept in a digital format. Companies, governments, and many other organizations also now focus heavily on computerized systems, and many organizations have migrated entirely (or almost entirely) to computerized systems, complete with digital storage of vast amounts of data and assets.
While there are many benefits to information being kept in a digital manner, issues are beginning to arise. Internet hostility, including hacking and malware are primary concerns, but another threat looms over every bit of digital information – namely, digital decay. We’ve put together this comprehensive guide to help you understand, and possibly prevent, this problem from affecting your digital assets and valuable data.
Digital decay is a term that is used to describe any degradation, obsoletion, or breakdown of computerized information. All digital storage is prone to some sort of decay for a number of reasons. This decay can also be known as “data decay” or “data rot.”
Digitized information is often more accessible and convenient than traditional hard copy methods. Instead of keeping documents physically on file, they can be condensed and kept on computers, external hard drives, disks and even by using cloud storage.
Physical files are in danger of elemental problems. For instance, pages yellow and unforeseen circumstances (i.e., fires, floods) threaten buildings where files are kept. Unfortunately, digital caching methods can be more vulnerable than physical caches. Files, pictures, and other data can be made unrecognizable.
The Scope of Digital Storage
There are two primary factors producing much of the digital decay problem facing the world. First, the rapid technological advances in storage hardware and methods. Secondly, the rate at which information is being digitized. The scope of digital storage is vast and getting bigger by the second.
A Brief History of Digital Storage
1950s: Storage devices like Drum Memory, Uniservo, and the IBM 350 were used to store information. Only a few megabytes (MB) per device could be kept.
1970s: The cassette tape and original 8-inch floppy disks were introduced and could hold up to 1.2 MB per small item.
1990s: Technology broke the gigabyte barrier with hard drives and increased storage on discs. The 3.5-inch floppy was introduced.
2000s: SD cards, Solid State Drives (SSDs), and the cloud has put infinite storage capabilities into the hands of millions.
Thousands of different devices and tens of thousands of software programs have been created to store, read, and apply data.
The Amount of Data is Increasing Exponentially
In 2013, the amount of data in the world was estimated at 4.4 zettabytes (4.4 trillion gigabytes). By 2020, the digital banks will hold 44 zettabytes. That’s ten times the data. To put this staggering number in perspective, one zettabyte would currently take up about 20% of Manhattan (source).
More capable devices and software have been created out of necessity to accommodate the exponential digital growth.
For a bit of reference:
A Megabyte can store about 800 pages of text
A Gigabyte can store nearly 900,000 pages of text or over 300 digital images
A Terabyte can store nearly 350,000 digital images
How Digital Files Degrade Over Time
Information is stored much differently, but there are illustrations available which can help us understand specific issues that create decay in digital information. Physical information (papers, books, etc.) degrade due to erosion. Depending on the conditions surrounding files, they may last longer or become illegible sooner.
Digitized information suffers from degradation of binary code (the coding system using 0 and 1). Updating systems and software can wear the stored files, causing them to degrade slowly. This corruption is different from something like a catastrophic failure. Viruses, physical damage, and other massive failures destroy data.
To illustrate the difference, we can use two examples of physical data. The Library of Alexandria held much of the world’s knowledge over 2,000 years ago. Tens of thousands of scrolls all legible and kept in one place. It was burned to the ground, and all of the information was lost. This was a catastrophe that we can liken to a large computer database facility being erased overnight.
Hieroglyphics have been used for many millennia and, in many cases, still remain today. However, most of these glyphs have eroded for hundreds of years and are far less legible. On top of legibility concerns, there are very few people in the world who can interpret what was carved or painted. This illustration provides a better understanding of how digital decay occurs and how files degrade over time.
Specific Causes of Digital Decay
Primarily, there are three causes for data to become corrupted, unusable or left behind over a span of time. We will go over each in detail.
Three primary causes of digital decay:
Hardware Advancements
Software Updates and Changes
Internet Information Changes
Hardware Advancements
A look at the number of device changes in our digital history would alarm many. For example, data stored on a Uniservo cannot easily be read today. Most people don’t have a means to see information on a 3.5-inch floppy disk or even a disc drive for more modern CD-ROMs.
Finding a way to read this information is only one step of the digital preservation puzzle. Once it’s consumable, data needs to be transferred and updated to more modern computation devices. Doing so, in some cases, will result in lost or damaged data.
It’s the difficult, dangerous and costly hardware transitions that have many government agencies and companies using now antiquated technology. For instance, according to CNBC, the US Military still uses 8-inch floppy disks to coordinate nuclear operations.
Software Updates and Changes
The only digital change happening faster than storage devices are software updates and changes. There have been many more thousands of software programs created to read and apply the stored data. Programs that can’t correctly hold and read information is a form of digital decay called versioning.
One of the best-known software products in the world is Microsoft Windows. Versions of this software are used by millions the world over. There have been hundreds of different software products under the name “Windows” along with thousands of updates and versions. Sometimes, new updates cause issues.
Issues with other software, problems with your computer, or even issues with your data can occur. In 2015, upgrading from Windows 7 or 8.1 to 10 would mean losing certain programs altogether. Not to mention, watching DVDs would require a “separate playback software” going forward.
Upgrades and versions can slowly change the way files are read and portrayed until they become unreadable or lost (either fully or partially).
Internet Information Changes
A very pervasive form of digital decay happens entirely online. At the time of this writing, there are over 1.3 billion websites and nearly 5 billion web pages. All of these sites are creating new information, and in some cases, new ways of storing and reading that data.
Sites often rely on third-party hosting to remain online, creating another weakness in data storage. When web pages are not working, hyperlinks to those pages become “broken.” Links (like those in this article) help source information and allow for further research. Pages that are no longer available creates a nuanced form of digital decay.
Also commonly known as “link rot,” it’s not just businesses and informational websites that suffer from this form of digital decay: According to a 2013 report by researchers at Yale University, a surprising number of websites cited by the Supreme Court are non-functional nearly one-third (29%) of the time. And according to Visual Capitalist, nearly half (49%) of all links in Supreme Court decisions are now broken. As these citations are crucial for judicial opinions and law review articles, link rot creates a substantial challenge for legal researchers. The fact that this is an issue that impacts the highest court of the land illustrates the pervasiveness of digital decay.
In addition to websites, social media is also a major source of file storage. Photos, memories and even videos are stored on the profile pages of billions of people. Imagine trying to recover your data if those sites change or go away entirely. Or, for companies, attempting to obtain videos and graphics from your social marketing campaigns.
These types of digital decay already occur. Myspace launched back in 2003, quickly rising to become one of the most popular sites on the Internet with more than 75 million active users at its peak. Of course, this popularity did not last, and as the platform lost millions of active users, Myspace was forced to pivot and is now a vastly different platform than it once was when in its prime (today, it’s a platform for musicians and artists). Unfortunately, this change caused data to be lost or damaged.
From the help center of Myspace:
*If you don’t see any photos, that means your old account was not synced to your new Myspace. Try doing a search to see if you can locate your old Myspace account. Unfortunately, if you cannot locate your old profile we will be unable to assist with retrieval since the old Myspace was never transferred to the new Myspace.
Potential Solutions to Preserve Data
Digital decay has been on the minds of many for quite some time. Art galleries trying to preserve art in a computerized form, university libraries, and older organizations have all felt the sting of file corruption.
Many choose to put a hold on hardware and software upgrades until it can no longer be avoided. Although, this becomes more expensive over time as old equipment becomes harder to find. There is also the increased threat of hacking and other issues that require regular updates.
However, there are three potential solutions to slow the decay process for your information.
Manual Transfer
A labor-intensive method of data preservation is to transfer files manually. When new hardware, software, and connection types become available, there are often ways to bridge the old and new. There is a fairly lengthy process involved.
For example, retrieving the data from a 3.5-inch disk. The disk drives are still available and, with a few adapters, connection to your current computer may be possible. However, your current operating system may not be able to read the data. Transferring data also doesn’t guarantee files will be kept perfectly intact.
Note: There are also paid services that specialize in transferring data stored on older devices into modern formats.
Emulate Older Software
Using emulators is a viable alternative to manual transfer. Software programs can be made to emulate older hardware and operating systems to run on more modern equipment. Potentially, the files stored in old formats will be kept and read in the exact same manner — ideal for avoiding digital decay.
However, there are eventual downsides to this method. The rapid advance of operating systems and hardware means an eventual need for new emulators. The process of developing these can be costly, and finding them isn’t always simple. Still, emulation is less cumbersome than manual transfer and allows data to remain relatively uncorrupted.
Future Universal Emulation
The most effective solution may not be created yet, but could be on the horizon. In 2005, IBM and others created the Universal Virtual Computer (UVC). Appropriately named, UVC is a very sophisticated emulator. This computer is able to read and render data from nearly any digital storage device.
To test the UVC’s capabilities, one group of researchers tested formats from 1960’s supercomputers all the way to their current computers (2012). This research, which you can find here, also explains the process in detail. At the moment, this computer does not appear to be available for commercial or personal use. The uses and testing have been promising.
Best Practices to Prevent Data Decay
Digital decay is one of the world’s fastest growing problems, and it hasn’t been solved just yet. There are promising solutions currently in practice and development. For now, individuals and companies are still left with data that is prone to many of the problems discussed.
The methods to permanently preserve your data and move it from one format to another may not be feasible. Despite this, there are steps to take that may hold off the decay as long as it can be held.
Quality Equipment: Better metal components and higher quality electronics make a difference for the digital information being stored. The better-made equipment is, the longer your files are likely to last —intact.
Multiple Formats: A benefit of rapid technological advancements is overlapping solutions. There are high capacity hard drives, cloud storage, and other devices. Backup data in multiple formats on multiple devices to increase the longevity of vital information.
Backup Online/Social Data: Instagram photos, live videos, even important social posts may need to be stored in multiple formats to ensure their survival for the long haul.
Adapt Quickly: Once a prevailing technology becomes dominant (think Blu Ray versus HD DVD), it may be a good time to switch. There will be more solutions (i.e., adapters, software) to aid in the transfer of your data.
Utilize Archiving Services: The Internet Archive’s Wayback Machine is perhaps one of the best-known archiving solutions around. This service has been around for more than two decades and has created permanent archives of more than 310 billion web pages at the time of this writing. Another helpful archiving solution is Perma, which focuses primarily on academia, journalism, and similar verticals that rely heavily on the use of citations and references in their work. Like the Wayback Machine, Perma creates a permanent archive of the web sources cited in assets and publications, ensuring the integrity of the work.
Leverage Digital Asset Management Solutions: One contributor to digital decay is the sheer volume of digital assets today’s organizations create and manage – which, over time, often become lost in the deep recesses of the myriad asset repositories typically used by companies and individuals. Leveraging a digital asset management solution such as MerlinOne provides a single source of truth for an organization’s digital assets, improving discoverability and streamlining rights management to eliminate digital decay caused by lost or forgotten assets.
Long-term protection of digital storage isn’t yet a perfected process. Until there are vital answers to our problems, preserving data will take a concerted effort by users. Following these best practices and training users on the appropriate processes and practices that reduce digital decay goes a long way in preserving digital assets and data for modern organizations.
Additional Resources on Digital Decay
To learn more about digital decay and how to avoid the negative impacts of this pervasive problem for your company, visit the following resources:
