2

If I have two RAW files, how I can test whether they are indeed the same but with merely different tags?

(By "the same" I mean "the same data from the sensor".)

If it matters, the format I'm thinking about is ARW.

mattdm
  • 143,140
  • 52
  • 417
  • 741
user541686
  • 646
  • 1
  • 5
  • 16

3 Answers3

3

Use dcraw with the -D option to extract just the non-interpolated pixel data from each file. This will result in a .pgm file — a grayscale map — which you can then compare (using them cmp command in Linux for bytewise compare, or by making a checksum, or whatever else). I think this should be exactly what you want, since all metadata is discarded, and the format is very simple.

It's possible that two different programs which generate pgm files would use a slightly different format resulting in a different output (since there can be whitespace and comments), but I'm pretty sure that the results from dcraw will have nothing that varies other than actual differences in image data.

mattdm
  • 143,140
  • 52
  • 417
  • 741
  • Slightly confused. If I get back a grayscale map, what happens to the color data...? (I probably misunderstood what you're saying.) – user541686 Aug 10 '15 at 01:38
  • Interesting approach... I took the question as just wanting to determine if both files were generated from the same file, but with metadata changed by software. If it's really the image data you want to compare, this makes sense. Would it be possible for the gamma correction to mask some differences, though? – junkyardsparkle Aug 10 '15 at 01:41
  • 2
    @Mehrdad It's a raw file. There is no color data as such. This outputs the value of each photosite completely disregarding the bayer (or whatever!) filter. – mattdm Aug 10 '15 at 01:44
  • @junkyardsparkle I don't think so. From the dcraw docs, By default, dcraw writes PGM/PPM/PAM with 8-bit samples, a BT.709 gamma curve, a histogram-based white level, and no metadata. I guess could add -W to fix the white level, but for the same image the histogram-based approach should always give the same reasult. – mattdm Aug 10 '15 at 01:47
  • If you do want interpolated results, you could use -h or one of the various -q options instead of -D. The practical result will really be the same, assuming (and I am) that the algorithms used are deterministic. – mattdm Aug 10 '15 at 01:55
2

I grabbed the NEX-3 RAW sample from here, made a copy of it, and ran md5sum on both: same checksum. I then used exiftool to add a Subject tag to the test copy:

 exiftool -subject="testing 123" test_copy.ARW

and did the checksum again, different this time. I then ran

 exiftool -all= *.ARW

to remove all tags, and again generated checksums. They once again matched. This should be applicable to your situation, using any binary checksum utility. Make sure you work on copies of the files you're testing - exiftool makes backups by default, but you shouldn't rely on this.

junkyardsparkle
  • 5,404
  • 16
  • 29
1

One approach would be to compare the images in Photoshop:

  1. Load both images into Photoshop, using the same raw conversion settings.

  2. Copy and paste one, so you end up with them as two layers on the same document. You can then flick between the two by toggling the top layer's visibility to pick up any significant differences.

  3. Change the blend mode on the top layer to "Difference". That will give you black pixels where the two layers are identical.

  4. A brightness or curves adjustment will help you spot subtle differences.

I'm sure you could do something similar in any decent image editing program.

You could also try using some sort of file comparison tool that can handle binary files on the raw files themselves, but interpreting the results could be tricky.

Adam
  • 224
  • 1
  • 4
  • This doesn't help. I'm looking to see if there are any differences, not just visible differences. And binary comparison doesn't help either, it's not like people memorize the specifications of each RAW format they handle... – user541686 Aug 09 '15 at 22:30
  • You could always write a custom piece of software for the task, if that's what you really need. – Adam Aug 09 '15 at 22:53
  • @Mehrdad provided you work in a 16-bit space this ought to work - any changes you make to a 14-bit RAW file will show up in a 16-bit conversion. You can make any binary difference visible by using the levels/curves tool. The only downside to this is that some RAW converters discard edge pixels so the image you get is marginally smaller than the actual number of pixels on the sensor. – Matt Grum Aug 24 '15 at 09:26