10

I have a collection of JPEG photos, each 500 to 600 pixels on the longest side. How can I detect which ones have been algorithmically enlarged from a substantially smaller photo?

An online marketplace requires each seller to upload photos of products that it sells, and these photos must be at least 500 pixels wide or 500 pixels tall because product photos with little detail cause a poor experience for buyers. I can already tell if a seller is trying to circumvent this requirement by adding a solid-color border, such as extending the standard white background with more white. But lately, sellers have started to circumvent this by upscaling old photos taken before the 500-pixel requirement was published. What is a good way to determine whether photos have been enlarged with nearest-neighbor, bilinear, or bicubic interpolation?

Community
  • 1
Damian Yerrick
  • 361
  • 2
  • 13
  • Just to clarify, are you asking about completely automated detection, without any evaluation by human eyeballs? – junkyardsparkle Feb 12 '15 at 19:10
  • 4
    For an automated test, stackoverflow is more likely to give you an answer – Renan Le Caro Feb 12 '15 at 19:14
  • @Renan Even a manual test using a sequence of GIMP filters could work for a spot-check of a particular seller's images. – Damian Yerrick Feb 12 '15 at 23:38
  • 1
    I guess the question we need to ask is why do you care? It makes quite a difference to the answer... – James Snell Feb 13 '15 at 09:10
  • @JamesSnell Bad photos repel buyers. See for example eBay's guidance. – Damian Yerrick Feb 13 '15 at 16:00
  • You can try taking the Fourier transform of the image and looking at the presence of high frequency components. Upscaled images won't have much of high-frequency components. Beware though that JPEG compression also removes some of these. I just tried this method and it seems to be fairly sensitive to down- then up-scaling. It will take a fair bit of work to make it reliable though. – Szabolcs Feb 13 '15 at 16:46
  • @tepples - that doesn't explain why you care and what you're actually trying to protect yourself from, or what you're doing (and what you have control of) all of which are things we'll need to actually give you a useful answer. – James Snell Feb 14 '15 at 13:12
  • @JamesSnell I assume the marketplace is trying to A. protect buyers from having to return a product whose blurry photo hides significant flaws in the product, or B. keep the UX consistent from one product listing to another. Whether my personal role is that of the seller, the buyer, or the marketplace ought not to matter because all three sides need to know what outcome to expect when a buyer reports a seller's habit of using blurry photos to the marketplace. – Damian Yerrick Feb 14 '15 at 15:12
  • Still... create barriers, expect circumvention. Especially from customers. – rackandboneman May 13 '19 at 15:22

5 Answers5

14

Have a DOG sniff out blur in the photos.

If you're going to be penalizing for digitally enlarged photos, you might as well penalize for out-of-focus photos too. The blurred edges and details in both cause the same bad experience for viewers, regardless of whether it is caused by a small original or poor focus. What you want to do is detect blur, which is an absence of high spatial frequencies.

Try taking the difference between an image and a blurred copy of itself. If an image is already blurry, a 1-pixel Gaussian blur isn't going to change the image as much as if the image were sharp. So there will be more difference between a sharp image and a blurred version than there is between a blurry image and a further blurred version. In computer vision, this technique is called the "difference of Gaussians" (DOG).

  1. Open the image in GIMP or another layered photo editor.
  2. Duplicate the layer.
  3. Apply a Gaussian Blur with a radius of 1 pixel to this new layer.
  4. Change the layer mode to "Difference". The image will go black except for the edges.
  5. Repeat steps 1-4 for a known sharp image of similar subject matter, composition, and size.
  6. Compare the intensity of the edges in the two difference images. You can eyeball this or use a histogram.

I just tried this on a 400x480 pixel photo and on the same thing that had been reduced to 200x240 (50%) and then enlarged back to 400x480 (200%), and the edges in the upscaled photo were quite noticeably fainter. It won't be conclusive on a mild enlargement such as 140%, but it will catch blatant cases.

Several computer vision libraries include means to calculate difference of Gaussians on an image. So do many graphical image editors. Recent versions of GIMP, for example. include a DOG macro that automates steps 2 through 4: Filters > Edge-Detect > Difference of Gaussians, then set the radii to 1.0 and 0.0.

Related questions on other Stack Exchange sites:

DOG won't catch nearest neighbor, but you can do that by looking for a pattern of rows and columns that are identical to their immediate neighbor toward the top or left.

  1. Open the image.
  2. Duplicate the layer.
  3. Offset the new layer one pixel up or to the left.
  4. Change the layer mode to "Difference".
  5. Look for a pattern of blank lines.
Damian Yerrick
  • 361
  • 2
  • 13
  • 1
    What if the upscaled image has a really strong unsharp mask applied? – mattdm Feb 13 '15 at 04:37
  • 1
    @mattdm unsharp mask increases contrast, it doesn't create high frequency image content. – Matt Grum Feb 13 '15 at 10:34
  • @mattdm USM is a high-boost filter: x + amt*(x - GB(x, r)). Combining USM and enlarge will boost only the mids (middle spatial freqs), not the highs, because the highs don't exist. DOG(x, 1, 0) isolates the highs. – Damian Yerrick Feb 13 '15 at 15:56
1

I do not that this is possible in the general sense. There are many possible upscaling algorithms, with a signature that may be difficult to detect unambiguously without knowledge of the image content (as an extreme example, an upscaled area of uniform colour is still uniform colour...).

Possibly an option would be to calculate a metric for image complexity, such as an entropy estimate (eg see https://stackoverflow.com/questions/4935380/get-or-calculate-the-entropy-of-an-image-with-ruby-and-imagemagick).

If you do this over a large number of images, you can generate statistics for the whole collection. You could then manually review images that are outliers in those statistics.

Unforunately, this is always going to result in false positives and images that have been up scaled well may not be caught (but if they are good, does it matter?)

Community
  • 1
Mark Moore
  • 51
  • 1
1

Actually you can

You don't need a dog to sniff the picture. Go to:

http://rest7.com/image_upscaled

On this page you can upload your image and will get original dimensions, like this:

{
  "is_upscaled": true,
  "current_width": "2000",
  "current_height": "928",
  "original_width": "1750",
  "original_height": "696",
  "accuracy": "82%",
  "accuracy_width": "82%",
  "accuracy_height": "82%",
  "success": 1
}

Sometimes it doesn't guess the original resolution correctly. I think it depends what up-scaling algorithm was used on the photo. Also I discovered that if a photo was upscaled and then compressed to a JPEG format with heavy compression (like 30%) the JPEG artifacts make it harder for this page to guess. But if your photos are of good quality, upscaled using popular methods (Lanczos, Bilinear) it should be quite accurate.

Here are 2 sample images:

ORIGINAL

https : // i . stack. imgur . com / iXYKV.png

(sorry, I don't have enough reputation to post more than 2 links)

UPSCALED & CROPPED

enter image description here

If you post the cropped photo this page will return:

{
  "is_upscaled": true,
  "current_width": "700",
  "current_height": "300",
  "original_width": 352.33333333333,
  "original_height": 151,
  "accuracy": "57%",
  "accuracy_width": "57%",
  "accuracy_height": "57%",
  "cropped": true,
  "success": 1
}

So you can see it detected the image was enlarged and cropped. It won't tell you size of the original image before cropping since it's just not possible to get information about something that was deleted just from the pixels that are left intact.

Jack
  • 11
  • 2
  • That site is only going to work as long as you can trust the data embedded in the image. But it is trivially easy to modify such data. And if I was trying to deceive a market place for money, I would be altering data left right and centre. – Peter M Jun 17 '17 at 13:49
  • @PeterM I am not quite sure what you mean. You want to crop an image? This site will still detect that it was upscaled. – Jack Jun 17 '17 at 14:20
  • Ask yourself how it knows that the image was cropped. Where do you think the information is stored that tells it what the original image size was? – Peter M Jun 17 '17 at 14:44
  • @PeterM Well, I am not sure how exactly it works but I used it a bit and I can tell you it analyzes the pixels and detects original resolution based on that. It doesn't analyze the file format but the pixels itself. So original resolution is not stored anywhere. – Jack Jun 17 '17 at 14:48
  • @Jack When "it analyzes the pixels", it probably uses something like DOG. – Damian Yerrick Jun 17 '17 at 15:07
  • @DamianYerrick Perhaps but doesn't require Gimp and less manual clicking. Can you estimate how accurate is your method with DOG? – Jack Jun 17 '17 at 15:10
  • -1 until someone tells me what kind of magic is going on, I'll assume it to be impossible to know the original size of a image that was cropped and scaled and edited and had metadata removed or changed. Pick a "random" small image from here and test it. I did, and the site was actually quite honest saying "accuracy": "0%" in its reply – Rolazaro Azeveires Jun 18 '17 at 09:06
  • @RolazaroAzeveires Of course it will not give you dimensions of the original image before upscaling and cropping. This is not possible. But it can tell you the image was upscaled even if you croped the image. I updated my answer to show you an example. – Jack Jun 18 '17 at 13:00
  • I removed my "-1", still this is a poor reply, because it relies on "magic" – Rolazaro Azeveires Jun 18 '17 at 14:35
  • 1
    Based on the similarity of the "accuracy" percents computed this site seems to be using a variant of the default algorithm in https://github.com/0x09/resdet -- I say that because this number is fairly specific and represents the percent of symmetric sign changes around a given row or column in the frequency domain (which when over 50% is a strong indicator that the image was upscaled from that location.) – tab Nov 17 '18 at 05:41
1

I'd take a hybrid approach. I think the other ideas of using a Difference of Gaussians, checking EXIF or other metadata, or even FFT can be combined. Another possibly easier way is to simply take each image, downscale it, upscale it again and compare. If they are very similar (using something like Delta E, perhaps), then it's likely that they were upscaled (or blurry as another post suggests). Perhaps you could make a threshold of the number of tests passed vs. failed? If more than half the tests pass, then it's good, otherwise, it's bad, or needs human intervention to verify, or something like that.

user1118321
  • 2,893
  • 15
  • 20
0

You should be able to do a good enough job by partially unpacking the JPEG data itself and doing some trivial counting.

JPEG data is created by performing a discrete cosine transform on the original image data, quantizing (throwing away the high-resolution data), then walking through the resulting DCT block in a zigzag pattern and packing the resulting stream of bits with Huffman coding.

If you reverse the Huffman coding and undo the zigzag, you'll have a series of 8x8 DCT blocks, in which the lowest frequency data is in the upper left corner of the block and the highest frequency information is in the bottom right.

What that means is that you can literally glance at the data in that intermediate format and tell whether it was upsampled, because all the 8x8 blocks will have nonzero values only in the upper left corner (roughly).

dgatwood
  • 2,235
  • 11
  • 15