If an image is rotated losslessly, why does the file size change?

Question

I was searching for methods of lossless image rotation and happened upon this question which explains it rather nicely:

Are "Windows Photo Viewer" rotations lossless?

So I created a 256×256 JPEG with random pixels (Photoshop cloud filter) and then rotated it using Windows Picture Viewer. After rotation, the file size actually increased, but only on the first rotation. Every subsequent rotation thereafter, the file size remained static. I know it's rotating losslessly because I've rotated it multiple times with no noticeable quality loss whereas a 257×257 image being rotated 20 times became very lossy.

Answer 1

This is most likely caused by entropy coding, which is the final lossless stage of JPEG compression, after the image data has been quantized to reduce its size.

When a JPEG image is losslessly rotated, this final lossless encoding layer must be undone, the unpacked DCT coefficients shuffled around, and then the shuffled coefficients need to be entropy coded again. Since the efficiency of the entropy coding layer depends on the order of the DCT coefficients within each block, which rotating the image will change, it should not be surprising that the rotated image file may be a few percent smaller or larger than the original.

There are also several different ways in which the entropy coding step can be done, so it's quite possible that the file size of the exact same JPEG image may vary depending on the software that does the encoding. Some of the potential differences between encoders include:

• choice of arithmetic encoding (rare but potentially more efficient, used to be patented) vs. Huffman encoding (simpler, standard);
• choice of sequential (each 8x8 pixel block is encoded one at a time) vs. progressive (low frequency components of all blocks are encoded before the higher frequency components, usually slightly more compact) encoding order;
• choice of using the standard Huffman symbol tables (faster, simpler, may be more efficient for very small images) vs. custom tables optimized for each image (usually more efficient for large images, slower and more complex to encode);
• if custom Huffman tables are used, different encoders may potentially generate different tables for the same image data;
• various low-level details of the encoding process itself, such as whether and when to include restart markers in the data stream, may also vary between encoders.

Also, the "JPEG files" people normally work with actually contain JPEG-compressed image data wrapped in a JFIF or an Exif container, which combines the image data with one or more metadata blocks, and introduces its own set of complications. Even if the software that rotates the image does not actually make any substantive changes to the JFIF/Exif metadata, simply rearranging the data could potentially affect the file size by a few bytes.

In particular, the JFIF/Exif metadata may contain one or more thumbnails of the full-size image, and software that rotates images really should regenerate (or also losslessly rotate!) the thumbnails so that they will match the new orientation of the full-size image. This alone could easily account for the observed size difference.

Answer 2

I went ahead and repeated the experiment to see if I could figure out what's going on.

Procedure

I generated a random 256-by-256 pixel RGB image using the "Solid Noise" filter in GIMP (Filters > Render > Clouds > Solid Noise...) using the default settings (shown below):

And the result:

Then I saved the image as a JPEG using the default settings:

Then I transferred the image to Windows and opened the image with Windows Photo Viewer by right-clicking on the image in the File Explorer and choosing Preview from the menu. Then I rotated the image using the buttons in the bottom, and saved the image by navigating to the next image using the arrow keys.

For each of the tests below I started with a copy of the original image, and rotated (clicked on the rotate button) the corresponding number of times before saving. Here are the reslting sizes (ls -l -r):

                    size in bytes    last-modified date 
                          VVVVV        VVVVV
-rwxrwx--- 1 root vboxsf   6258 Nov  8 11:24 original.jpg
-rwxrwx--- 1 root vboxsf  23645 Nov  8 11:30 cw.jpg
-rwxrwx--- 1 root vboxsf  23636 Nov  8 11:30 cw-cw.jpg
-rwxrwx--- 1 root vboxsf  23649 Nov  8 11:30 cw-cw-cw.jpg
-rwxrwx--- 1 root vboxsf   6258 Nov  8 11:27 cw-cw-cw-cw.jpg
-rwxrwx--- 1 root vboxsf  23649 Nov  8 11:31 cw-cw-cw-cw-cw.jpg
-rwxrwx--- 1 root vboxsf  23649 Nov  8 11:29 ccw.jpg
-rwxrwx--- 1 root vboxsf  23636 Nov  8 11:29 ccw-ccw.jpg
-rwxrwx--- 1 root vboxsf  23645 Nov  8 11:29 ccw-ccw-ccw.jpg
-rwxrwx--- 1 root vboxsf   6258 Nov  8 11:27 ccw-ccw-ccw-ccw.jpg
-rwxrwx--- 1 root vboxsf  23649 Nov  8 11:30 ccw-ccw-ccw-ccw-ccw.jpg

Immediate observations

• Windows Photo Viewer (WPV) does increase the size dramatically; the amount of increase is around four times in this test!
• All the new images increase to around the same size, but they are not identical.
• WPV does not re-encode or even re-save the image when it is rotated by a multiple of 360 degrees. (The timestamp, 11:27, is when the files were first copied.)

Using cmp -l on files that should have identical content allows us to see where the files differ.

robert@unity ../jpeg-rotate-test % cmp -l cw.jpg ccw-ccw-ccw.jpg
 2223  63  62
 2224  60  71
 2226  60  64
 2227  60  66
robert@unity ../jpeg-rotate-test % cmp -l cw-cw.jpg ccw-ccw.jpg
 2223  63  62
 2224  60  71
 2226  60  64
 2227  62  64
robert@unity ..jpeg-rotate-test % cmp -l ccw.jpg cw-cw-cw.jpg
 2223  62  63
 2224  71  60
 2226  64  60
 2227  61  64
robert@unity ../jpeg-rotate-test % cmp -l cw.jpg cw-cw-cw-cw-cw.jpg
 2221  60  61
 2223  63  61
 2224  60  66
 2226  60  61
 2227  60  61
robert@unity ../jpeg-rotate-test % cmp -l ccw.jpg ccw-ccw-ccw-ccw-ccw.jpg
 2223  62  63
 2224  71  60
 2226  64  65
 2227  61  64

These files differ in only four bytes (actually in a timestamp), meaning that WPV is doing the same thing every time; now we only need to figure out what that is.

Detailed Observations

For this I used JPEGsnoop to see what exactly was is in the images.

Since the outputs are fairly long I've linked to them as a gist. Here's a summary of the differences:

• GIMP uses only an APP0 (JFIF) and COM (comment) segment for metadata. WPV leaves the APP0 segment untouched, but curiously adds a null byte to the comment (so that it is null-terminated).

• WPV adds two APP1 segments, which are Exif and XMP metadata. These segments are 4286 and 12726 bytes, respectively. Together they account for almost the entire increase in filesize.

• GIMP produces a progressive JPEG, while WPV produces a baseline (non-progressive) JPEG. For this reason GIMP's image has multiple scan segments, while the WPV image has only one. In my experience the progressive image is sometimes slightly smaller.

• GIMP used 1×1 chroma subsampling, while WPV used 2×2 subsampling. This leads me to believe that WPV is not using "true" lossless rotation, unless it is somehow able to detect this is a black-and-white image.

To resolve these issues, I ran a second test.

Procedure

I followed similar steps to the first test. I created a random 256×256 RGB image using the RGB Noise filter (Filters > Nose > RGB Nose...) with the following settings:

Here is the result:

I exported the file as JPEG using the following settings:

Progressive has been turned off, but Subsampling is still set to 4:4:4 (which is another name for 1×1 subsampling). The quality is increased to 98.

I copied the image and rotated the copy clockwise; then copied the rotated version and rotated that copy counterclockwise, so that we can directly compare the quality between the original and the WPV processed copy.

Results

-rwxrwx--- 1 root vboxsf 159774 Nov  8 16:21 original-random.jpg
-rwxrwx--- 1 root vboxsf 222404 Nov  8 16:24 cw-random.jpg
-rwxrwx--- 1 root vboxsf 222467 Nov  8 16:24 cw-ccw-random.jpg

Although the increase this time is smaller in relative terms (around 40%) the absolute increase is even greater—around 62 kB. This suggests that WMV is using a less efficient encoding.

I'll use ImageMagick to compare the two images:

robert@unity ../jpeg-rotate-test % compare -verbose -metric AE original-random.jpg cw-ccw-random.jpg null:
original-random.jpg JPEG 256x256 256x256+0+0 8-bit sRGB 160KB 0.000u 0:00.009
cw-ccw-random.jpg JPEG 256x256 256x256+0+0 8-bit sRGB 222KB 0.010u 0:00.010
Image: original-random.jpg
  Channel distortion: AE
    red: 0
    green: 0
    blue: 0
    all: 0
original-random.jpg=> JPEG 256x256 256x256+0+0 8-bit sRGB 0.050u 0:00.020

There are zero pixels different between the original and rotated copy. So, even if WPV is not using "true" lossless rotation, it is doing a good enough job. I suspect I know what is going on, and to explain I'll divert a little bit into the math behind JPEG compression.

The JPEG compression algorithm breaks an image into 8×8-pixel blocks. Each one of these blocks is then subjected to a Discrete Cosine Transform (DCT). The resulting DCT coefficients describe the block as a sum of different-frequency waves. The algorithm then "throws away" some information in the high-frequency waves that correspond to noise and very small detail. The decoding process reverses the DCT, adding the stored waves together to get back the block.

It is possible to rotate the DCT "waves" without actually undoing and redoing the transform (basically you turn all the horizontal waves into vertical waves and vice-versa). What I think happens in WPV is that the image is actually decoded, rotated, and then re-encoded. During the re-encoding process, since our image's size is a multiple of 8 in both dimensions, each of the new blocks corresponds to one of the original blocks. Importantly, since each block has no high-frequency components, the algorithm doesn't throw away any information, and finds exactly the right DCT components that a "true" lossless rotation would have.

Lastly, I'll look at the components of the JPEG files again. The results are again linked as gists. Comparing the two:

• The WPV image contains an extra 4286+2 bytes of Exif metadata, 1 extra byte in the comment, and 12,726+2 bytes of XMP metadata. This is a total of 17,017 bytes of additional metadata. What is all that data used for? I peered into the file with my trusty hex editor and a copy of the relevant standards:

  • Exif metadata is structured like a TIFF image, which contains a number of tags (there's way more complexity, but I'll skip right over it). Most of the bytes in the Exif segment are contained in two identical tags with tag number EA1C (59,932 decimal). That tag number is not documented anywhere I could find. Both tags contain 2060 bytes of "undefined" type, which are all null bytes except for the first six (1C EA 00 00 00 08). I have no idea what these tags are, why there are two of them, and why they need to be 2 kB each.

  • The XMP metadata is actually an entire embedded XML document with namespacing and long UUIDs, that just contains the WPV version string (that was already in the Exif metadata). However, that only accounts for around 400  bytes. The remainder of the segment is 122 repetitions of 100 spaces followed by a newline. That's over 12,000  bytes of totally wasted space.

• Like the previous test, both GIMP and WPV use the same DCT quantization tables. This means that they should be calculating the exact same DCT coefficients, which is why the images are exactly the same. I'm not sure if WPV just happens to be using the same quantization tables or if it copies the tables from the input.

• Unlike the previous test, this time WPV uses 1×1 subsampling, so it may actually be detecting that this is a color image (or at least that higher samples are necessary to losslessly re-encode the image).

• GIMP and WPV use different Huffman tables (part of the entropy coding step). The tables for WPV are larger by a total of 279 bytes, and in one case contain 7 times as many codes.

  Looking at JPEGsnoop's statistics, we can see that some of these codes are rarely used. For example, in the ID: 1, Class: AC table, of the 119 16-bit codes defined, only 23 are actually used. Overall, the actual scan segment is 28.5% larger in the WPV version.

Summary

• WPV may not be doing "true" lossless rotations, but the rotations seem to be practically lossless.

• The extra size is partially due to a fixed amount of added metadata, and partly due to less efficient entropy coding.

Version Information:

• OS (Linux) (uname -a):

  Linux unity 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux
  

• OS (Windows):

  

• GIMP (Linux): 2.8.14 (from package gimp, version 2.8.14-1+deb8u1)

  

• Window Photo Viewer (according to image metadata):

  Microsoft Windows Photo Viewer 10.0.10586.0
  

Answer 3

^{EDIT: This answer was posted before I knew that the files had increased in size by around 9 KiB (9055 bytes for the 256×256 image, 9612 KiB for the 512×512 image).}

In all likelihood, when you first rotated the image, Windows Picture Viewer did one (or both) of the following things:

1. Added an EXIF tag that was not in the original JPEG image (perhaps the Orientation tag);
2. Modified/added information to a tag that already existed (perhaps Processing Software or Image Software tags).

This increased the file size because of the additional EXIF tag (and/or additional data to existing tags).

Subsequent rotations didn't increase the file size because all of the tags and/or tag data that WPV would have added/modified were already there. Only the Orientation Tag value changed (and perhaps date/time tag values as well).

---

EDIT: It is almost certain that this explanation cannot account for about 9 KiB of additional data in the file. Further, absent any other reasons for the size increase, this explanation would expect that the increase in size would be more or less constant (modulo some length differences between string representations of numeric data, probably a few bytes). That is obviously not what is happening here, at least not the complete explanation.

Answer 4

Without reverse engineering the jpeg en/decoder it's impossible to say for sure. There are actually a number of jpeg standards and contrary to popular belief, not all of them can be modified without re-encoding.

It's possible that the first save is a lossy rewrite into its favoured jpeg flavour and the subsequent rotations are a simple metadata tweak or an operation directly on the DCT table (which is possible for some encoding schemes).

The increase in filesize may also include some additional metadata, though 9k seems a lot, it's possible. The increase may also be accounted for by the addition of a thumbnail which may not have been present in the output from GIMP. We may be able to get more information from the files directly (before WPV and after).

In any case, trying to work loslessly with jpeg is really a fool's errand as it's only useful with certain image sizes, not all decoders and encoders are identical and it requires those editors to work directly with the jpeg content which you cannot rely on to be the case... Just because it does so now does not mean it will continue to in the future.

Your better bet is to work with a lossless format and avoid the pain entirely.

Answer 5

Lossless JPEG rotation is only possible without introduction of boundary artifacts if the image dimensions are multiples of the block size (typically[/always?] 8). See the jpegtran man page (sorry I don't have a good canonical link for it; feel free to edit if you find one) for details on what's involved:

The transpose transformation has no restrictions regarding image
dimensions. The other transformations operate rather oddly if the image dimensions are not a multiple of the iMCU size (usu- ally 8 or 16 pixels), because they can only transform complete blocks of DCT coefficient data in the desired way.

jpegtran's default behavior when transforming an odd-size image
is designed to preserve exact reversibility and mathematical
consistency of the transformation set. As stated, transpose is
able to flip the entire image area. Horizontal mirroring leaves any partial iMCU column at the right edge untouched, but is able to flip all rows of the image. Similarly, vertical mirroring leaves any partial iMCU row at the bottom edge untouched, but is able to flip all columns. The other transforms can be built up as sequences of transpose and flip operations; for consistency, their actions on edge pixels are defined to be the same as the end result of the corresponding transpose-and-flip sequence.

For practical use, you may prefer to discard any untransformable
edge pixels rather than having a strange-looking strip along the
right and/or bottom edges of a transformed image. To do this, add the -trim switch:

I suspect the Windows Photo Viewer is avoiding this issue by performing decompression and extreme-high-quality recompression to simulate lossless behavior when the image dimensions are not multiples of 8, rather than actually performing lossless rotation. A good utility would just do actual lossless, artifacts and all, or drop a few pixels, rather than ruining the quality of the whole image (and increasing the file size).

Answer 6

I have no definite answer but some possible theories of why that happened. Some file types work in such a way two different codes for an image of that file type don't necessarily produce different images. For example, the PNG file type works in that way because it allows for a transparent background but an image with a transparent background and one that's the same except that the same background is white appear exactly the same way. An image file is said to be compressed if it takes up less than 3 bytes of memory per pixel. I believe that except for the ones with a transparent background, no two PNG files generate the exact same image. When ever you save an image as PNG, it converts it into a code that generates the original image and except for very unusual images like one where each pixel is a random color of all 2^24 colors, the code will take up less memory than 3 bytes per pixel so saving as PNG is said to be lossless compression. On the other hand, to save memory, only certain images can be generated by the code of a JPEG image file. There's probably more than one JPEG file type and I don't know if any of them have the property that two different images of that file type can generate the exact same image. I'm assuming that a bunch of times you just rotated an image then saved it as JPEG and will gives an explanation of what happened under the assumption that that's what you did which I don't know if is true. A rotation you did is lossless if there's a way to get back the exact same image file code as you had before you rotated it and saved it. You might not be correct that you really did do a lossless rotation. If it really was lossless, one possible reason is that two different image codes really do generate the exact same image and it was the code of the image you had and not which image you had that determined the image you got after you rotated it then saved it.

Answer 7

The reasons behind this are a few

the way images are coded and compressed will change the size simply because of the compression algorithm. you can test this by saving it as a bitmap and then rotating it. In that format or any raw format the size should remain the same. If it does not then the program saving the image is adding new data possibly some metadata or something.

But why are you rotating a jpeg 20 times?

Answer 8

Because of how compression of images works. Any format like PNG or JPG in general do not preserve file size after rotation.

To the compressor the rotated image is just a different image, due to how compressione heuristic works there's no guarantee it will compress a rotated image the same.

Of course if the compression is lossless, if you rotate the image 4 times the 4th time the image is again the same (rotated until it becomes tilted as original): in that case it should become again of the same compressed size, if not then it is because one of the following reasons:

• Added metadata: the program added some chunk of text for some reason
• Compressor changed: the program may choose to just re-save the image as the original if there are no changes, but if you apply any change (even 4 rotations of 90 degrees) it may decide to re-compress the image again using its own compressor (the program no longer knows it is still the same image).
• In general the same compressor (libPNG or libJPG) yields very different results across different implementations, different versions of same library and with different compression parameters (also operative system and compiler makes a difference here sometimes).

Image compression works by compressing images into 4x4 or other sizes chunks. In general a compressor sees a rotated image as a different image, however since a compressed pixel chunk is Just a linear decomposition, if the chunks on the image are the same it is possible just to Transpose/Mirror the linear decomposition matrices effectively keeping the same quality:

Note this has to be implemented on a per feature basis, and that also explain the initial increase in size => on first rotation it just trys to compress the image in chunks that are rotable:

• If it fails to do so: the image quality degrades

• If it is successfull it increase the size only once then every rotation keeps same quality.

• That operation is successfull only if the image is made by equal chunks. (size of image is multiple of chunk's size).

The scottbb answer ins wrong and you can do a simple test:

• Open the original image: Screenshot it
• Rotate the image 4 times with WPV: Screenshot it
• Compare the 2 screenshots

You will see the image changed (it is recompressed on first rotation). However that change is limited in time, you can now rotate it again without quality loss (if the image has a size that is a multiple of 8)

To directly answer OP:

I know it's rotating losslessly

Not it is not rotating losslessy, it lose quality at least once (on first rotation: because it should first compress it in a way that can be rotated), then it maintains its quality.