12

I was having a conversation with my father who used to be a photographer, and I said “when I open a raw file…”

He interrupted me and said, "you can never open a raw file, the editing software simply creates a “preview” of the raw file."

I told him, since the invention of “raw editors” like Lightroom and camera raw in Photoshop, you can certainly open raw files, the software simply shows a visual representation of the raw data and makes local copies. But you are still opening the raw “file”. Anything with a raw extension like Nikon's .NEF or Sony's .ARW is by definition a raw file.

My guess is that my father is getting confused by how software worked before “raw editors”. As far as I understand, the software actually created a physical JPEG preview, but now raw editors actually open raw files without the need for a jpeg preview.

I’m sure it’s a case of linguistics but I’d be keen for somebody knowledgeable to clarify whether you can actually open “raw files” or not and maybe explain a bit more about how raw editors work as opposed to their predecessors.

scottbb
  • 32,685
  • 12
  • 104
  • 188
Daniel K
  • 129
  • 1
  • 3

10 Answers10

33

It's linguistics.

Of course you can open a raw file, as the term "to open a file" has a very broad spectrum of different, but related meanings.

  • On a low level, "to open" just means to read the bytes in the file, and leave it to some software to do with them whatever the software wants to do. You can do this with any file in any format.
  • On a higher level, "to open" means to open and "understand" the bytes to an extent that a useful, intellegible representation can be shown to an end user (e.g. the text from a Word document shown in the correct fonts and sizes). This needs software that knows how to interpret the bytes.

A raw file contains the data as they were read out of the sensor, in a vendor-specific format, in theory without any further processing. So:

  • On a low level, "opening" a raw file gives you the intensity-per-sensor-pixel values, which is not very useful to end users.
  • On a higher level, "opening" a raw file includes converting the low-level data to an RGB image that can be viewed on screen, or saved in e.g. JPEG etc. But converting from raw to RGB needs a lot of image processing and non-obvious parameters (white balance, sharpening, gamma curves etc.). So, when you "see" a raw file on screen, the result is influenced by many (arbitrary?) settings that have been applied in the process.

Whenever you "open" a raw file, you do not really see the true contents of the file, but rather one of many possible processing-based interpretations of the raw file. You can legitimately use the term "to open" for that, but need to be aware of the processing that has been done in between.

Ralf Kleberhoff
  • 1,269
  • 6
  • 7
  • 17
    Semantics, rather than broad linguistics [but in pointing that out, it makes me look a real pedant… semantics again;)) – Tetsujin Sep 25 '23 at 09:19
  • @ralf-kleberhoff not sure if you can read my reply, it wasn’t populating your name in the reply.

    So we got to the conclusion that my dad thinks that you can only “open” files which you can manipulate directly. For example like a JPEG. Since you can never edit a ‘raw file’, but you can only save the edits as a jpeg or other format. My argument was, you can certainly open it, but the software then interprets the raw data. It’s a very silly and circular argument but I just wanted to argue it out of principle. Now my father is saying I think I know better than everybody as always. Fml

    – Daniel K Sep 25 '23 at 12:22
  • 8
    @DanielK, I would say that your father's interpretation is more accurate. When you access (open) the raw data, the software used is only showing you a preview of one of the many different possible ways in which that raw data might be interpreted. It is extremely probable that different software will generate a different preview due to somewhat different demosaicing algorithms/default settings. And some software will not show you the raw data at all; only the embedded jpeg stored within it. – Steven Kersting Sep 25 '23 at 13:10
  • 5
    @StevenKersting you can argue the opposite as well. The same is done for JPEGs (just that we mostly agreed on how to interpret this data). You could also interpret it as text data and get a bunch of gibberish. Also why stop there? You cant "show" JPEG data either, it needs to go though a bunch of different processing steps to a frame buffer, though a VGA/HDMI/DP cable to a monitor which then shows this monitors representation of said JPEG. And we all know, these can VASTLY differ XD – Max Sep 26 '23 at 13:49
  • 2
    Also from a computer science standpoint, writing data back to a file is very easy (and possible). We just agreed that in case of RAW files, this might not be the best idea. – Max Sep 26 '23 at 13:51
  • 9
    @DanielK The definition "can manipulate directly" also fails for a JPEG file - since it is a lossy compression format, with a number of tuning parameters and optional features, very few modifications can be made without re-generating the entire file. Image editors will often make this explicit, distinguishing between "saving" the editor state in a proprietary format (e.g. PSD, XCF), and "exporting" to a new JPEG file (which might happen to use the same filename and overwrite the previous version). It's interpretation all the way down. – IMSoP Sep 26 '23 at 16:00
  • 7
    @Max The difference between JPEGs and the data in a raw file is that the JPEG has ONE singular way that it can be correctly displayed. The information in raw files can legitimately be represented by a near infinite different number of interpretations. – Michael C Sep 26 '23 at 20:12
  • @MichaelC No, that is just plain wrong (as is the argument in your answer). We just CHOOSE one way. There is nothing stopping me writing a JPEG viewer/editor which just doubles the intensity of all blue pixels. That is still a correct interpretation, just maybe not as close to reality. But that was never a claim of JPEGs anyways. Oh and in what world cant you do things like image manipulation to a JPEG? Ever heard of instagram? Are the pictures with filters on there somehow "incorrect" interpretations of the original JPEG? – Max Sep 27 '23 at 06:38
  • 4
    @Max: JPEG as an image format is a compressed encoding of YUV (or RGB) pixel data that has one correct decoding. And optionally a colour profile that fully specifies one unique mapping back to true physical colours, as Michael C commented on an answer about that RAW vs. JPEG difference. RAW files don't have this property, they record the raw sensor data and other parameters are needed to map that to true colours. – Peter Cordes Sep 27 '23 at 13:53
  • 1
    @Max You can write a program that does something else, but that's at least logically equivalent to decoding and then filtering. And yes you can encode a new JPEG that represents a modified image, obviously. One fun thing you can do without fully decoding a JPEG is to losslessly rotate it 90 or 180 degrees: this involves decompressing the lossless Huffman coding (the final stage JPEG uses, a bit like a ZIP file), and then just transposing the DCT coefficients of the 8x8 blocks and re-doing lossless compression. Never decoding back to the spatial domain and then redoing DCT + quantization. – Peter Cordes Sep 27 '23 at 13:59
  • 1
    @PeterCordes The point still stands. You CAN open a RAW file and see a correct decoding of the content (i.e. pixel intensities). You can also apply a near infinite amount of parameters to manipulate the way you display a JPEG. It's just that you choose to arbitrarily place your definition of 'correct' in such a way, that both file formats fall on opposite sides of that definition. – Max Sep 27 '23 at 15:03
  • @Max Here's what an unprocessed decoding of raw pixel intensities looks like. There is no standardized set of instructions to get from that to any kind of color image. There is a specific standardized set of instructions for how to display the data within a JPEG file. If you choose to change the way a JPEG looks and export it, the data contained in the file you export is NOT* the same as the data contained in the file with which you started.* You must alter the image data. This is not the case with different raw interpretations. – Michael C Sep 28 '23 at 23:39
  • @Max "Ever heard of instagram (sic)? Are the pictures with filters on there somehow "incorrect" interpretations of the original JPEG?" Yes. Yes they are. The actual image data has been altered to get them to look that way. – Michael C Sep 28 '23 at 23:45
  • 1
    @Max The internationally recognized standards for displaying JPEG images are ISO/IEC and ITU-T JPEG specifications. Published as an international standard in 2011 by ITU as Rec. T. 871 and, in 2013, by ISO/IEC as ISO/IEC 10918-5. – Michael C Sep 28 '23 at 23:50
  • @MichaelC I'm a programmer and I regularly work on image sensors. I regularly look at unprocessed RAW image files (sometimes while they are still more or less energetic electrons on a piece of copper), write RAW image files and edit them to find pixel errors. Fundamentally there is no difference between RAW and JPEG files: There is one perfectly well defined representation of a RAW file, pixel intensities (I's just not the one YOU want at this point). At the same time, I can take a JPEG file, "edit" it and save the editing steps in a different file (without altering the original data). – Max Sep 29 '23 at 06:48
  • Unless you're looking at a bunch of 0s and 1s, you're not looking at the actual raw data, you're looking at a rendering of that data that is dependent upon how that data is interpreted to create an image. The fundamental difference is that there is a standardized way to convert the 1s and 0s in a JPEG file. There are no such standards for raw files. In fact, there are not even such standards about how the data in a raw file is written. It varies greatly from one camera maker, or even model, to another. Your sidecar "edits" are not compliant with the standard of displaying the data in the JPEG. – Michael C Oct 01 '23 at 06:31
15

It is all already said, I just want to make a meme and a philosophical exercise.

Here is a JPG file opened... on a program that does not make a "preview" of the file.

enter image description here

I prefer one that actually makes a viewable view of the image file I am opening.


When we open a file, are we really opening a file? Or we are reading the content and making a useful interpretation of the content by the proper software, so we can see, read, and understand.

Rafael
  • 24,760
  • 1
  • 43
  • 81
  • 1
    I really like this answer for how obvious it is. If you want to pedantic, you can be in your own world view not in other peoples ;) – Max Sep 26 '23 at 14:21
  • 2
    I like this answer too. And to really blow your mind, note that the first four bytes mark it as a RIFF file, since JPEG is a compression method not a file format. The files were really JFIF and now commonly RIFF. The question focuses on RAW, but all files must be opened and interpreted in this manner. Nothing happens except in memory and the disk storage is a detached representation of memory along with tools to reconstruct a context. One could make a file format that consisted of stacks of pennies. – Yorik Sep 26 '23 at 18:22
  • 11
    This looks like a WebP file, actually. But the point you're making holds true regardless. – mpontes Sep 26 '23 at 23:03
  • WebP is a sub-type/is based on RIFF. "The WebP container (that is, the RIFF container for WebP)" ( https://developers.google.com/speed/webp/docs/riff_container#riff_file_format ) – Yorik Sep 27 '23 at 17:59
  • 3
    (Guys, it is supposed to be non-readable, now you are analyzing it more than the photo itself) n_n – Rafael Sep 27 '23 at 19:47
  • 1
    I really hate this answer because it's completely missing the point of JPEG-files: There is one single standardized view of how to display a JPEG file. There is no "preview" and your screenshot is a definitely WRONG view. Yes, the file is technically open, but the part about "previews" and so on is flat out wrong – Hobbamok Sep 28 '23 at 10:20
  • 1
    Let's go deeper into the philosophical view. If you zoom out, you have a different (pre)view. If that zoom-out has no aliasing, it is different. Even if you view your image you are not viewing the file. It is a representation of a file that is stored on your hard drive. If you are viewing it on the back of your camera you are not seeing the complete image. I can go on and on... I could change the file I used as an example, and open some RAW files so you are not hurt by me using the JPG file. But it does not really matter. – Rafael Sep 28 '23 at 11:30
11

You can open a RAW file the same way you can open a JPEG file. Both need to be interpreted by software to show useful data. You might say that the RAW file has many different interpretations, but the same is true to a degree for JPEG as well! E.g. some viewers might not honor the color profile.

ths
  • 7,161
  • 1
  • 11
  • 25
  • 10
    There's only one correct interpretation of a properly encoded JPEG that includes a color profile. Viewers that ignore color profiles are showing faulty interpretations. – Michael C Sep 26 '23 at 20:14
  • 1
    Except that JPEG files have a very narrow range for different interpretations. There's usually exactly one, which is the opposite for RAW – Hobbamok Sep 28 '23 at 10:21
10

I told him, since the invention of “raw editors” like Lightroom and camera raw in photoshop, you can certainly open raw files, the software simply shows a visual representation of the raw data and makes local copies.

Raw editing/conversion applications do give A representation of the data in a raw file on your screen. But we must be careful to not think they give THE representation of the data in a raw file on your screen. They don't.

In that respect, your father is correct. The data in a raw file must be processed to present anything remotely resembling a viewable image. How that data is processed and presented on screen varies greatly from one application to another, and even within the same application depending on what default rendering settings are selected when the raw image file is first opened. That's why opening the same raw file using different applications can give radically different results.

Raw image files contain enough data to create a near infinite number of interpretations of that data that will fit in an 8-bit jpeg file.¹ Anytime you open a raw file and look at it on your screen, you are not viewing "THE raw file." You are viewing one among countless possible interpretations of the data in the raw file. The raw data itself contains a single (monochrome) brightness value measure by each pixel well. With Bayer masked camera sensors (the vast majority of color digital cameras use Bayer filters) each pixel well has a color filter in front of it that is either red, green, or blue.² For a more complete discussion of how we get color information out of the single brightness values measured at each pixel well, please see RAW files store 3 colors per pixel, or only one?

How the image you see on your monitor when you open a raw file will look is determined by how the application you used to open the file chose to interpret the raw data in the file to produce a viewable image. Each application has its own set of default parameters that determine how the raw data is processed. Since each application uses a different set of instructions to process the raw information contained in the file, each result will be different.

One of the most significant parameters is how the white balance that is used to convert the raw data is selected. Most applications have many different sets of parameters that can be selected by the user, who is then free to alter individual settings within the set of instructions used to initially interpret the data in the raw file.

¹ Sure, you could take a picture that contains a single pure color within the entire field of view. But most photos contain a wide variation of hues, tints, and brightness levels.

² Except the "red" filter is really more of a yellow-orange color, the "green" filter is more a yellowish-green color, and the "blue" filter is a violet-tinted blue color. In other words, the colors of the filters in a Bayer mask do not correspond to the three colors our RGB monitors emit and blend to reproduce the response in our retinas that many other colors do. In fact, the colors of the filters in a Bayer mask are much closer to the three colors that each of the three types of cones in our retinas are most sensitive to than they are to the three "primary" colors we use for our RGB color reproduction systems.

Michael C
  • 175,039
  • 10
  • 209
  • 561
6

This is just semantics.

In computing, you don't really open a file, you read the file. And what does it mean to read a file? It means that (at least some of) the data in the file is read into memory, and the computer interprets that data and usually does something with it.

Yes, applications can read raw image files, and they can do something with the data in the file. That will involve some interpretation of the data in the file in order to display a preview on your screen, create some intermediate representation of the image for editing, write an interpretation of the raw image in JPEG format, etc. That hasn't changed since cameras started making raw image files available.

I think what your dad is getting at is simply the fact that raw image files are not really image files in the traditional sense – they require processing to convert the data in the file to a traditional image file. So we can't open a raw file as an image file per se, without processing it in some way first. As long as everyone understands that, then it's not really useful to argue over the precise terminology that people use.

See this question:
What does an unprocessed RAW file look like?

osullic
  • 12,093
  • 1
  • 23
  • 47
  • 6
    Actually in computing you typically do open a file (mainly for efficiency reasons) for later processing, just like you open a can ;-) Reading (or writing) would follow the opening then. – U. Windl Sep 27 '23 at 06:41
  • 1
    Opening a file (in computing terms) has not much to do with efficiency, but is required for completely different reasons. On most systems, when opening a file, the program requests read or write access to the file and specifies in case of write access if a new file should be created, an existing file overwritten or appended to. When opening the file, the operating system will verify if the requesting process has the required access rights to the file and in case of write access, often impose a lock on the file to prevent multiple processes from writing simultaneously. – jarnbjo Sep 28 '23 at 12:09
2

Of course you can open a raw file.

If your father corrected you in that regard, it looks like he is trying too hard to appear smarter than he really is.

[Open file is] a file, typically a disk file, that has been made available to the application by the operating system for reading and/or writing.

Source: https://www.pcmag.com/encyclopedia/term/open-file

If you want to read the contents of a .PDF document, you can open it for reading with a variety of software tools. However, editing said file is not as easy, is it? Would your father consider such action as "not really opening" that file, since you cannot edit the contents? If so, most of regular PC users would disagree.

As far as I understand, the software actually created a physical JPEG preview, but now raw editors actually open raw files without the need for a jpeg preview?

The preview being created is neither raw nor JPEG, it is just a data sequence stored in the volatile graphics memory section. The specific format of that image depends on your graphics card or APU, and it is neither JPEG nor raw. Meanwhile, JPEG and raw file format are two different ways of storing said image in non-volatile memory of the device. From those two, the format being stored in video RAM is much closer to the raw file format, though, as JPEG file format is compressed and mangled, thus the sequence of its bits cannot be simply copied to the video memory and be expected to form a sensible image.

2

You cannot properly view a RAW file, because the colour depth information is 12 bit or 14 bit, and the preview will typically be down sampled to only 8 bit. Even if you had a monitor that could match the bit depth of your RAW file, you would also need an OS that could make use of it, and also editing software. This doesn't exist yet, as high bit depth standards haven't been widely adopted for images. That might change one day, but we're in a crossover period where most things still downsample to 8 bit, so right now, you cannot truly open a RAW file.

When you do "open" a RAW file in image editing software, you then edit it to choose which information you're going to keep and throw away. You final result will be a downsampled 8 bit representation of your RAW file, and the results of this can vary wildly, depending on what information you keep, and what information you discard.

My point being, the default downsampling when you open a RAW file is very unlikely to be a good representation of that file. So the question is, are you truly opening a file if you cannot see an accurate representation of it?

Doctor Jones
  • 141
  • 4
2

To be even more pedantic about it, there is no correct way to open a RAW. There's also no correct way to open a JPEG or any other image format either.

But there is a standardized way to open a JPEG that is clearly defined by some party (in this case, the International Organization for Standardization) that everyone (or most everyone) chooses to adhere to: https://jpeg.org/jpeg/. However, because RAW formats tend to be proprietary, there is no public standard to produce or handle a RAW format outside of what a publishing company provides or how a third-party interprets the format. There is no universal standard for RAW format, hence multiple formats existing and each having multiple interpretations (i.e., previews) depending on the software used to display (i.e., open the file, read the data, and push it to a monitor) them.

Soulis
  • 121
  • 5
1

You have to define open. It is very simple to open a raw file on a low level. For example, with Python:

raw_photo = open("path_to_raw_file", 'rb')
raw_photo.read()

And there you have it, you've opened the file.

Now, you can choose to display it how you want - you can display an exact representation of the raw file and let you manipulate the raw data, or you can make a less accurate preview in the JPG format, which is better for quicker edits, but results in a loss of quality as you are working with a compressed image.

So not knowing anything about photography I don't know how the different photo editing software work, but on a technical level it is possible to open a raw file, and I would assume that older software used a JPG while newer software lets you 'open' and manipulate the actual raw file.

Kovy Jacob
  • 111
  • 1
0

Using any program which interprets raw image correctly is certainly "opening raw file".
If somebody means that you must see the raw image data to claim that you opened it there's https://www.rawdigger.com/.

Euri Pinhollow
  • 5,071
  • 17
  • 36