I have had discussions over and over again about “how should I backup my pictures”? And I am not talking about just a few pictures you sitting on your computer, I am talking about gigabytes or even terabytes which contain pictures from (paying) customers. In this blog post I will try to explain what risks there are, how you can protect against these risks, and what NOT to do.
Negatives in the Digital Age
In the analog days, you used to have physical negatives. You would not care about the prints that came out, you’d just store the negatives. Protecting these negatives from for example a fire was hard; you just had the one physical negative of each shot.
In the digital age we still have negatives. Every serious photographer will take his or her photos in the RAW format, except for some who sell straight out of the camera (still – in this case the JPGs are the negative).
Normally, you’d take the RAW files, and post-process them useing workflow software like Apple’s Aperture, Adobe Photoshop Lightroom or Corel Aftershot Pro (previously known s Bibble) which I like to use. All of these products have one thing in common: They leave the RAW (negative) untouched, and create a small file besides the negative that holds the settings for “printing” to JPG. These settings contain information like exposure, contrast, color temperature adjustments etc. The workflow software finally delivers output TIFFs or JPGs which can be considered the digital prints (which you could have printed on paper).
Having digital negatives has two huge advantages:
- Because the negatives are digital, they will never degrade through time; as long as you are able to keep the RAW files, the quality will remain the same, even after 1000 years;
- Because the negatives are digital, you CAN make perfect copies of the negatives and store them in different places.
As you can see, having digital negatives has huge advantages for long-term storing. You need to be able to accomplish one thing though: Manage to NOT LOOSE them!
All storage devices WILL break (at some point)
The first basic rule in storing images, is recognizing that every storage device will eventually break. No matter where you store your digital negatives, every digital media will break at some point. So how to survive these failures? Some ground rules:
- Store your data knowing that the media will break at some time;
- Make sure your data survives if the device breaks;
- Make sure you store your data on different physical locations;
- Know when your storage device starts failing or has just failed.
The first point seems relatively simple; as long as you have multiple copies you’d think you are fine. But beware of the 4th point: If you do not know your storage device has failed, there will be no action taken to “reprotect” (eg make another copy have multiples again). This is very much underestimated by a lot of people who still think backing up on DVDs is a pretty neat idea. To make sure you understand what I’m saying: Backing up on writeable DVDs is about the worst you can do.
So how to cope with these ground rules? Following these rules, there are some do’s and some don’t s. In the next paragraphs I’ll describe all technologies that are around to store our data, inluding their do’s and don’t s.
Different types of storage for your pictures, and their individual dos and don’t s
In this section I will be describing all storage types I’ve ever heard of, and their individual do’s and don’t s.
Storing your data on a hard disk
This is done often. In the non-professional environments I think this just might be the most popular way to store photos. Local hard disks are easy accessible from your computer, they are relatively fast (fast enough for workflow software). Another plus: They can be BIG (3TB is no longer extreme for a single hard disk these days). The downside? Hard disks will fail. Often, the operating system of a PC is also installed on the same drive if the disk is used as an internal drive, and when the operating system crashes, people often think they lost their photographs as well. Very often though, you can just connect your “crashed” drive to another PC, and recover your photos without issue, or maybe with the help of recovery software such as BadCopy Pro from Jufsoft.
Hard disks generally wear if they perform seeks (the head moving over the platter). The drive spinning is not really a factor for wear. Also, spinning up and down of these drives is (no longer) a reason for wear; the very old hard drives (the 10, 20 and 40MB drives way back) did wear when they went through a lot of spin ups and downs).
Storing your data on a digital media card (CF or SD-card)
Although I have never seen a digital storage card loose data over time, the relatively small size of these cards usually make them unsuited for long-term backups. In general, flash cards will wear as they’re are (re)written to, so adding photos can pose a risk for the other data on the card (although you normally would be able to recover the old data easily).
Storing your data on a Solid State Drive (SSD)
A Solid State Drive (SSD) is basically very much like the digital card, only with a different physical interface (mostly SATA) and a somewhat bigger size (you can get 256GB devices for decent prices these days). These devices are very fast when it comes to random reads (eg. booting your OS), but are less effective in transporting sequential data (like large RAW files). They are about as effective as rotating internal hard disks in this respect. SSDs also suffer from wear just like the digital media cards.
In general, both hard disks and SSDs have about the same life expectancy.
Storing your data on a CD, DVD or BlueRay disc
CDs, DVDs and more recently BlueRay discs are still a very favorite medium to backup data. In my opinion, it is the worst medium to backup to:
- Burned disks can have a very short lifespan; chances are that a 5 year old DVD has become completely unreadable;
- Burned disks degrade faster when they are stored in hot or moist environments;
- Burned disks scratch very easily;
- There is no way of telling how long a burned media will continue to be readable when read;
- Burning media discs is a manual process which means you need to be very structural in manually making backups;
- Burned discs are very easily affected by moist (flood/leakage) or heat (fire).
I will go deeper into these what we call “offline backup media” in part II (as you may have gathered, they will not be part of any acceptable solution though). Eventhough there are some vendors who claim their burned media will survive for 1.000 years (like the M-disc from Millenniata) , there are still many disadvantages (item 2, 4 and 5) so I would not recommend them as well.
Storing your data on a digital tape
Digital tapes where used for years and years by professional companies as their primary and only way of backups. Looking back at that time, most companies have or are in progress moving away from tape to disk-based backup solutions, and with good reason. Tape is, just like burned media, an offline storage medium. This means the tapes sit somewhere, usually in an offsite location, and you have no idea if those tapes will be readable when you get them out again. Also, a lot of tape standards suffer from not being readable by just any tapedrive, but there are variations in head positions that can pose serious problems. Modern tape standards like Linear Tape-Open (LTO) do not suffer from this issue btw.
Storing your data in the cloud
What’s this “cloud” thing? The cloud, as far as storage is concerned is just some type of storage (without any provider hard disk-based) sitting somewhere in the world. Via the internet you can access a small part of this storage, almost like the digital version of a storage box in a Shurgard or Citybox location.
In general, these storage facilities are located in datacenters somewhere in the world. They all have protection against hard disk failures (the “how” on this will be explained in the 2nd part of this blogpost), and it is not uncommon that they actually replicate data between sites or even continents (replication is a technology where all daat written to one location is automatically synchronized on another location). You can usually upload and download your data via client software, and usually also web-based for single files or small groups of files. Data sitting on your computer is usually automatically looked at, and uploaded automatically when it is not yet present in the cloud. This software is usually very smart; if you have the same file twice, it will only send it one single time. This is called deduplication, and has several forms which are less relevant for this conversation. Important is that the deduplication is source-based, meaning the data is not even trasmitted if found on the central cloud storage already. This will save you a lot of bandwidth.
When you decide on backing up into the cloud, your initial sync will proobably be huge. I myself have around 550GBytes of negatives, and it took me 3,5 weeks to perform the initial sync. That may seem like a logn time, but as I gradually add negatives now (1-4Gbytes for a shoot at a time), the software syncs up in less than 5 hours and I’m safe.
Stay tuned for part two!
In this first part I have described possible separate media to store your data and backups. In the second part I’ll focus on surviving failures of a single storage medium by combining them, and also failures of an entire set of storage mediums. I will slowly work towards a solution that is reliable, affordable and I think a perfect solution for 99% of the use cases.
Update: Part 2 of this series is available here.