24

I've recently started to use RAW with Lightroom. I'd like opinions on how I should make sure my images are readable (not necessarily editable) a few years hence. Heck, let's say a decade some decades.

Should I keep my archives in Nikon's raw format, in conjunction with the LR database? Should I convert them to TIFF? How about DNG?

mattdm
  • 143,140
  • 52
  • 417
  • 741
gerikson
  • 4,270
  • 1
  • 23
  • 37
  • There is some related discussion at http://photo.stackexchange.com/questions/62/is-switching-to-dng-worthwhile – Jari Keinänen Feb 01 '11 at 13:08
  • 3
    Digital photography has been around for a decade now; is there any raw format that has fallen by the wayside, that is no longer readable with up-to-date software? In other words, is there actual precedent that indicates file format obsolescence is a real danger in photography? – Lyman Enders Knowles Feb 01 '11 at 18:13
  • @Lyman I know I wrote 10 years above but @labnut's answer below looks beyond that. I'm not worried about JPG or TIFF being unsupported but some camera manufacturer's RAW format? Sure, I think they can be "forgotten". – gerikson Feb 01 '11 at 20:23
  • Another question - will anyone care about your pictures in 40 years? ;) –  Feb 02 '11 at 12:46

12 Answers12

25

Having finished scanning 40 year old film I can assure you that you need to think longer term than 10 years, in fact at least 40 years.

To know whether there is an answer one must understand the problem. These things can happen:

  1. proprietary software makers stop supporting old formats, very possible after 40 years.
  2. proprietary operating systems stop supporting old photographic programs, also very possible after 40 years.
  3. a copy of the 40 year old proprietary operating system will no longer run on current hardware, highly probable.

So, there is a real possibility that in 40 years time you will no longer be able to read your RAW images, using proprietary software. This is not to criticise proprietary software makers. Their shareholders require them to generate profits and growth which can be incompatible with maintaining decades old software.

Can anything be done?

  1. store your images as DNG. Support is coalescing around this format, making it rather more likely to survive over the long term.
  2. store a copy as a high res jpeg. This will be readable for a long time.
  3. keep a copy of your operating system and programs in a virtual machine. For example, for other reasons, I keep a copy of Windows 98 in a virtual box, allowing me to run it under more recent operating systems, on more recent hardware.

But, because I am involved in the open source world, I am confident there will always be an open source solution. I say this because:

  1. it is a requirement of the GPL licence to keep the source code available. This means you can always locate the relevant program and recompile it to run in the current environment (or somebody else will do it).
  2. there is an army of open source programmers who delight in supporting even the quaintest and most esoteric things. An example of this is that, right now, Linux supports a wider variety of hardware devices than MS Windows.
  3. open source programmers are very active in supporting the various RAW formats.

You may remember the Commodore 64. It was introduced in Jan 1982, 28 years ago, but was quickly obsoleted by the then new IBM PC. But even today you can run programs for that machine thanks to the Commodore 64 emulators developed and maintained by the open source world. This is evidence that we will be able to depend on open source solutions for a long time.

Michael Mrozek
  • 103
  • 3
labnut
  • 8,297
  • 3
  • 34
  • 55
  • 1
    Great points! I do think that as computing technology matures (arguably it has, already), large scale abandonment of platforms and formats becomes rarer. The point about virtual machines is interesting too, it' quite new as a technology for mainstream users. – gerikson Feb 01 '11 at 13:31
  • bump for fullres jpeg copy and software+OS in a VM for future compatibility! – JoséNunoFerreira Feb 01 '11 at 14:20
  • 2
    Labnut, not to go too off-topic but just to make sure it's said -- the GPL doesn't have any way to make sure the source code is really available, since doing so still requires someone who (a) has a vested interest in keeping it available, (b) has funds to store it somewhere accessible to the world, (c) has time to maintain the storage site, access to it, etc. So I'd stipulate that open source makes it potentially more likely that source code will be out there somewhere, but there isn't -- and can't be -- any guarantee of that. – delfuego Feb 01 '11 at 14:38
  • Gogo dcraw! (Re broad raw-format support.) I expect that dcraw will continue to support old raw formats long after the digital camera makers cease to support them. – C. K. Young Feb 01 '11 at 14:46
  • Great question, and a great answer. +1 for keeping a VM around. – D. Lambert Feb 01 '11 at 14:56
  • 8
    I'll spoil the VM-fun a bit by pointing out that VM-images are like RAW-images: proprietary formats. And even if not: nobody assures at all that general purpose systems will exist and be sold in the future. Might be that we are all stuck with a kind of super-IPad just because 95% of the people like a locked and defined system and vendors changed there. Could be a bit difficult too to get your hands on processors running without certain certificates in several decades. Prediction of computing is ... shady :) – Leonidas Feb 01 '11 at 16:35
  • @Leonidas: there are a wide variety of VM image formats and converters between many of them. Maybe raw disk images — which in this case means basically no particular format, just a data dump — are the safest. It's reasonable to assume that any system which can emulate a given physical hardware type can load disk images that match the hardware. But your point about the risk to general-purpose computing due to the super-ipad is well taken. – mattdm Feb 01 '11 at 16:41
  • 4
    It's much more likely you will be able to somehow read RAW format, rather than be able to run an old VM. – o0'. Feb 01 '11 at 18:02
  • 1
    On the "make sure it's said" front, there's also the question of whether or not you'll be able to provide power to -any- computer 40 years from now... If, say, peak oil issues and/or economic breakdown and/or some sort of geo-political meltdown happens, who knows what sort of power infrastructure might exist, let alone hardware vendors, etc. If we're even alive through who-knows-what, it's -possible- (not to claim it's -likely-) that our only real option would be to have made prints. Just something to maybe consider, depending on what your goals are, exactly. – lindes Feb 01 '11 at 18:22
  • @lindes: in that apocalyptic scenario, I bet your concern would be surviving the hordes of squatters, mutants and zombies; whether you are still able to access your old photos or not would be kind of a minor issue, I believe. You'd be happier if you can, sure, but that's beside the point. – o0'. Feb 01 '11 at 18:27
  • @looris: The point where it would be relevant is if you're a 50-years-later (or whatever) survivor, rather than a 50-minutes or 50-hours later survivor. It's when you're trying to teach your grandkids about the way things were that led up to the downfall that you'd want the images. At that point, it would be highly relevant to anyone who had managed to evade, repel, or (gasp) befriend the other survivors (let's not apply pejoratives and fictional creatures to this hypothetical, not sci-fi, scenario), if they still had (some portion of) their belongings, but couldn't power a computer. – lindes Feb 01 '11 at 22:53
  • @Leonidas - I've always looked at VMs as a kind of Matryoshka doll thing. When you move away from a platform, move it into a VM for long term storage. Then when you move away from the system you upgraded to, Move that into a VM as well. That way, all the software you need is packed up in a series of VMs. Sure you have nested VMs, but I don't think DOSbox on windows XP on Windows 7 would be much slower than DOS was originally. – Fake Name Jul 15 '11 at 08:22
  • There is no guarantee that software 40 years from now will be able to read DNG files generated by one of today's cameras. Each sensor model requires a different set of demosaicing algorithms. Just because a RAW file is placed in a DNG container (and possibly some of the original data stripped away), does not transform the actual sensor readout to a different format. Just as .nef or .cr2 files from newer cameras can not be read by older versions of software that don't support files created by that specific camera model, all DNG readers can't read all DNG files. – Michael C Nov 20 '13 at 07:56
8

Adobe's Digital Negative format is an attempt to create a free open "universal" format explicitly to solve the issue of long term digital archiving. It is used natively by some cameras but not yet by the big two (Nikon/Canon). It is however easy to convert from Nikon/Canon Raw to DNG. It make take a little more time to see if DNG will gain traction in the mainstream.

I would certainly keep the original RAW files and also convert to DNG to hedge my bets. I think Nikon's RAW will be easily usable in 10-20 years, simply due to the sheer number of images images in the world. 100 years? Maybe not.

Ken
  • 5,078
  • 4
  • 39
  • 56
  • 1
    DNG stores unprocessed data like the proprietary raw file - it contains (sort of) unprocessed sensor data, the format of that data is dependent on sensor technology - as technology progresses new types of data will have to be added to DNG (unless you think bayer arrays are the last word in sensor design), assuming DNG will still be around in 50 years you will have DNG files full of data in an obsolete format that has not been used for 30-40 years - believing raw processors will still support old DNG is the same as believing they'll support CR2 or NEF – Nir Jul 14 '11 at 13:35
  • As Nir hints at, just because a raw processor supports DNG files from one particular camera is no guarantee it supports a DNG file from another camera with a different sensor design. Just as older versions of ACR can't open .cr2 or .nef files from newer camera models, if a particular sensor's demosaicing algorithms are not contained in the DNG conversion software then it will not be able to open DNG files from cameras with that sensor. – Michael C Nov 20 '13 at 08:01
5

While file formats may become obsolete it is not going to happen in one moment. After new format is introduced software will continue to support old one for some years - so you will have plenty of time to convert all your photos. Also in worst case you can always install old software (and if software no longer works on your system you can use virtual machine) which supports your format.

Sergey Matvienko
  • 1,007
  • 8
  • 11
  • 1
    You assume that the owner takes care of his data all the time. Sometimes people die and sometimes their legacy is discovered late. – Leonidas Feb 01 '11 at 13:31
  • I must admit i have not considered this. However my point about using old software is still valid. – Sergey Matvienko Feb 01 '11 at 15:41
  • As the number of photos created per minute/day/year explodes into the billions, there will be fewer and fewer legacies "discovered". They will be lost in the glut of information, just as the details of the vast majority of countless billions of human lives are lost when that person dies. – Michael C Nov 20 '13 at 08:05
5

The file format is basically irrelevant

What's much more important is the physical medium. Look at seven inch floppies, three inch, qic, sun scsi, pata, ... All these and more have gone and it will be more and more difficult to get drives, and even if you have drives, the interconnects for them.

You will need to periodically update the physical media, then the files will almost certainly be readable by some piece of software or other.

time4tea
  • 151
  • 1
2

This is a good question, unfortunately I do not know a good answer. My guess is that you are safe as long as Nikon exists and as long as they do not force you to buy a new camera or software ;-)

However, since NEF, Nikons RAW file format, is proprietary and Nikon's software does not offer DNG export - as far as I know - you will not have a 1:1 copy of the RAW file in DNG format. Furthermore we do not know wheter DNG will exist that long or reach a wide compatibility and acceptance.

Thus what I am doing is, saving the final picture as a high res JPEG along with the RAW file. JPEG is around for a while and has, in my opinion, the longest expectation of life and acceptance from todays digital image formats.

el_migu_el
  • 720
  • 4
  • 11
2

To be honest, I think this is a non-issue and the RAW formats will be supported a long time in to the future. Canon/Nikon RAW formats may be proprietary but there are many programs that can read and convert them into other formats, and I think there always will due to the shear number of files as @Ken stated. I think I would be more concerned whether the backed up media will last that long and still be readable.

Look at computing today, a lot of old computers from the 80's/90's have been emulated to run on modern hardware - I am sure in 50 years time our quantum computers/Matrix will be more than capable of emulating today's technology. Unless we have a nuclear holocaust, in which case we won't be too concerned about whether or not we can read our old files and be more concerned about surviving and stopping our limbs from dropping off.

Piers Myers
  • 161
  • 2
  • 5
  • I did not address the importance of physical backups but of course I see them as very important in this case too. – gerikson Feb 01 '11 at 15:14
  • If we survive a nuclear holocaust, it might indeed be nice, once we're out of immediate danger, to have photographic evidence of "the way it was". I otherwise am inclined to agree with this answer. – lindes Feb 01 '11 at 19:10
2

I wouldn't worry about it for several reasons.

  1. The history of computing shows an extremely strong tendency for backwards compatibility and the status quo. x86, a 33 year old computing architecture, has actually gained market share over time in personal computing despite many superior competitors. As long as this is the case, old software and operating systems can easily be made to run on new hardware. There are very few programs written in the past 30 years that I can't run on a month old computer with little effort and cost.

  2. There has been an explosion in the number of RAW photographs. People are aware of this and do not want this information lost, especially because RAW formats are considered archival quality. People have learned from their past mistakes where proprietary formats have been forgotten.

  3. RAW formats are basic enough that they have been reverse engineered by anyone who needs to read them. What this means is that anyone who wants to display a RAW image can find open source software (which implicitly describes the format) that will do so. Even if through some catastrophic event this knowledge is lost (I would argue that opposite will happen: the major camera manufacturers will make the formats open in the future), people will be able to reverse engineer the formats again.

rm999
  • 1,746
  • 1
  • 10
  • 15
1

TIFF is far more timeless. In 10 (or 40 or 100) years it's just as likely software would be able to read your Nikon RAW file as a DNG file, so to expend the effort in conversion makes little sense to me as any conversion to a more homogenized format risks losing some data from the original file.

Conversion into 16-bit TIFF files is your best option, to preserve as much tonal range from the image as possible.

I am not suggesting you delete the RAW files, you should always keep your RAW files as they are your "digital negative". This is true regardless of whatever format you convert to. But TIFF as an alternate format will always be able to be read easily by pretty much any program that deals with images (even browsers can display a TIFF file).

Kendall Helmstetter Gelner
  • 3,442
  • 19
  • 20
  • Converting to regular TIFF surely has as much if not more potential for losing data as conversion to DNG (which is based on TIFF/EP, by the way, which in turn is based on TIFF - so really, a DNG file kind of is a TIFF file; and certainly DNG can be 16-bit (or more!)). Specifically, you risk losing data about the color filter array and/or other meta-data. I worry about this with DNG, too, to some degree, but at least DNG is specifically trying to address it. The problem is whether your converter understands and duplicates every last bit of data from the RAW file. I keep my RAWs, just in case. – lindes Feb 01 '11 at 19:00
  • 1
    P.S. My "just in case" statement is, of course, potentially in conflict with the stated reasons for converting - namely that software would stop supporting old formats. And that is a risk. To best mitigate both risks, convert but keep the original, too. Though that of course increases storage requirements. However, I'm also of the opinion that it would be asinine to remove from software such as Lightroom the ability to read any format it was ever able to read. But then, the Adobe marketing for DNG suggests they think differently about that. We'll see. – lindes Feb 01 '11 at 19:06
  • @lindes: I meant to say you'd keep the original RAW file + the TIFF. Some people convert to DNG and throw away the original RAW files, which is more what I was warning against... and since you'd be keeping the RAW files anyway I see no need to also have a DNG. – Kendall Helmstetter Gelner Feb 01 '11 at 19:59
  • 2
    If you mean to convert to TIFF and keep the RAW file, then I suggest you update your answer to reflect that. It seems like you're potentially suggesting a convert-and-delete-the-original approach, in which case I think (generic) TIFF is a much poorer option than DNG, because of all the meta-data that DNG files will strive to maintain. – lindes Feb 01 '11 at 22:42
  • Post has been altered to clarify thought. – Kendall Helmstetter Gelner Feb 02 '11 at 04:01
1

The only constant in life is change, so planning for that change is very important. I have some negatives I inherited from my grandparents that are over 60 years old (circa 1948-1949). These negatives are on a non-standard film (by today's standards), larger than 35mm but not quite medium format--127 format. My scanner can still read the format, although I have to be careful handling it.

You aren't going to find a single digital standard that is going to outlast the film, but there are still things you can do to hedge your bets.

  • Store your image files in a vendor neutral, no-loss storage format. TIFF files with 16 bits per channel (48 bits per pixel) will keep all the detail your camera can dish out. Most sensors don't do more than 12 or 14 bits per channel in RAW. Just be careful not to use JPEG compression in your TIFF files.
  • Update your stored files over time. Sometimes a change to the spec has to be made that is not compatible with older versions to fix new problems. By batch upgrading your digital files every 5 years, or every decade, you can stay current enough.
  • Be a ruthless editor. Before you update your stored files, take the opportunity to weed out the pictures that just aren't important enough to keep. Your daughter's birthday party may be an important event, but you don't need every single picture from it.

The pictures you have that survive 10, 20, 30, 40, 50, 60+ years should be the ones with the most importance. They can be historical, sentimental, or just really good timeless pictures. Some pictures only have a very limited time of significance. For example, I do sports photography at my alma mater high school, and the pictures I take only hold significance until the children in it graduate. That's when I bundle them up, and give them to the kid's family. (it's a small school).

Another option to avoid digital obsolescence is to avoid digital altogether. There are some services out there to convert your digital pictures to film. Depending on the emulsion, film can resolve up to 2400 lpi (line pairs per inch), or roughly the same as 4800 dpi (dots per inch). It's a big expense, so you should only consider it for the pictures you really want to keep long term. It also takes up a lot of room, but it is an option.

Berin Loritsch
  • 1,512
  • 9
  • 14
1

One photo file format with good longevity prospects is PNG. PNG files can store images with 16-bit-per-channel color depth and lossless compression. Unlike the raw formats, PNG is an open standard, implemented by freely available software libraries and supported even by very basic editors like Windows Paint. Unlike TIFF, PNG is a relatively simple format that's easy for programs to support fully.

PNG is largely overlooked in the photography sphere, but I suspect that its openness and simplicity will lead it to be supported for a long, long time to come.

Lyman Enders Knowles
  • 1,217
  • 8
  • 10
  • PNG does not support EXIF, sadly. – fmark May 24 '11 at 13:44
  • 1
    I just took a 12MB .NEF file (Nikon RAW) and saved it as a PNG. The resulting PNG file was 19MB, which is larger than the original RAW format. Even after extremely thorough PNG optimization through 3rd party compressors I was only able to get it down to 16MB. 12MB as RAW with all its benefits, or 33% larger as a PNG with none of the benefits of shooting RAW in the first place seems counter-productive. – Bryson Jan 04 '12 at 23:03
  • PNG is good for archiving significant artistic, documentary, and historical images. It is not so good for archiving huge numbers of mundane images due to the resources it requires. – Michael C Nov 20 '13 at 08:08
1

As long as open source software exists which can read it now, you'll be fine.

If your files can be read by open source RAW software such as dcraw (or software that depends on it, like ufraw of RawTherapee), then you can breathe easily. As open source, it can never be withdrawn by its vendor, and its source code is always available to others to build into new software, even when operating systems, file systems, hardware architecture etc change.

There is a big tendency for backwards-compatibility in open source (in general, all software, but especially open source). Open source comes with source code, and with source code you can compile for any operating system and hardware, even ones that haven't been invented yet, without starting from scratch. So even if the original creator of the code is no longer around, you can bet that others will still use the code and publish (free) software to use it on any hardware and operating system that's popular.

Once you have software that can read some RAW format, it's easy to adapt it to read any new RAW format. It doesn't require much extra code, as a lot can be shared. So there is no reason to remove support for particular RAW formats from software.

thomasrutter
  • 13,600
  • 2
  • 40
  • 53
0

In Lightroom 3, when you are importing your files and converting them to DNG you have an option to embed the original raw file. This means that your original RAW file will be inside of your DNG file.

So you can choose this option if you want long-time "compatibility" and also saving your original RAW file. But this means bigger DNG files.

cahuk
  • 219
  • 2
  • 2