0

Has anyone (besides Adobe) correctly reverse-engineered Nikon NRW RAW12 CFA data?

My mission is to provide 'natural', unprocessed RAW12 mosaic (CFA) source to test an image processing system. The ISP is meant to work directly on CFA data, so my intent is to get unmodified data as it comes from the sensor, as if I were reaching in and grabbing it from the MIPI CSI-2 link.

If I had my preference I'd be doing exactly that: grabbing CSI-2 data from a MIPI camera using Raspberry Pi or some such. I don't have access to that. Instead, I have on hand a sample from my Nikon B700, shot as RAW12 (uncompressed) mode, in an NRW file.

Here's what exiftool has to say about what's in the NRW:

enter image description here

To get my RAW12 test data, I extract the CFA binary directly from the NRW with hex editor using info gleanded from exiftool, as shown above. The CFA data is the correct size for the raw image (5200x3902) and format (uncompressed RAW12.) Then I snip out a 2Kx2K region of interest from the CFA data using a tool that I developed. The resulting file is (well, should be) a 2Kx2K RAW12 binary.

When I view this snipped CFA data in Irfanview as RAW12 packed-pixel (2 pix in 3 bytes), I get this dogs-breakfast image:

enter image description here

This implies that whatever Nikon does in its NRW file, it is not simple RAW12 packed pixel. Again, the question: Anyone know what the actual storage format is?

(Note: before you blame Irfanview, I independently tested its 'Open as RAW' and know that it works correctly on RAW8 and RAW12 packed-pixel, it can also view RAW16 too. I've also independently tested my RAW snipping tool.)

In the meantime, I did a workaround using Adobe DNG converter on the NRW. I snip out the DNG's RAW16 data with the hex editor and take out the same 2Kx2K region of interest with my RAW snipping tool. This gives the expected data, and here is the result:

enter image description here

That's fine, except the data are not the same as what came off the Nikon sensor. It's being reformatted and/or processed by Adobe DNG converter. So the NRW->DNG path is somewhat less useful to me for testing: I can't make the claim that I'm processing actual sensor data.

I suppose what I could do is get a camera that natively supports DNG?

Finally, one more thing: 'dcraw' gets it wrong, too.

hacktastical
  • 101
  • 1
  • Does this answer your question? https://stackoverflow.com/questions/68821340/raw-12-bits-per-pixel-data-format – Steven Kersting Dec 16 '23 at 16:10
  • @StevenKersting unfortunately no, it doesn’t. But it does give me an idea of possible thing to look into: endian-ness of the CFA data. – hacktastical Dec 16 '23 at 23:48
  • it is little endian. There is also a black point value in the exif that should be applied (DNG conversion fixes the black point using that value). – Steven Kersting Dec 17 '23 at 16:19
  • Is the black point value the only correction that DNG conversion applies, or are there others? At any rate, that doesn’t explain the weird result. – hacktastical Dec 17 '23 at 17:09
  • It's the only correction that is "baked in" in a raw>dng conversion... it also discards data from masked pixels that some raw converters can use, but that should not be relevant to your issue. When the DNG is displayed there is a WB correction applied as well, but that is not baked into the source data itself. – Steven Kersting Dec 17 '23 at 20:47
  • That's useful to know, so that I have some reasonable confidence that the DNG RAW data aren't too far from sensor values. Again, that doesn't explain the scrambling. The vertical pattern artifact appears to have a period of 8 pixels, which would hint that the data are being packed in a container that size. Perhaps the data are being sent as 2-d blocks to make de-mosaic more efficient? – hacktastical Dec 19 '23 at 21:04
  • Also, I've been working with a RAW14 NEF file, which doesn't seem to have this ordering: 14-bit data are packed into 16-bit containers, which I can pull out and view directly. – hacktastical Dec 19 '23 at 21:05
  • Correct, a DNG conversion is very close to being an exact raw copy (as opposed to dng output of some editors). I wonder if the striping could be photosites used for focus, which get less light and have to be computationally corrected for. But AFAIK, all Nikon mirrorless have the AF stripes arranged horizontally (vertical line sensing). And if it's not from a mirrorless that wouldn't be the case anyway. – Steven Kersting Dec 20 '23 at 13:30
  • I don't think the striping is related to AF. The striping has a periodicity of 8 pixels, which leads me to believe some kind of block encoding is being used, likely 8x8. Using this kind of encoding could make demosaicing more memory efficient since you don't have to have several full 'strides' for the filters. It could also be a compressed format of some kind, with compression set to a minimum to be allegedly 'uncompressed' as the EXIF claims. Anyway, I have the DNG workaround for now. – hacktastical Dec 21 '23 at 20:16

0 Answers0