Image Processing: How to detect a quadrialteral signboard in image?

Question

How can I detect a quadrilateral signboard from an image captured using a mobile phone? How can i detect shapes like rectangle? rounded rectangle (rounded corners instead of shape corners)?

I am using opencv.wrapper but i am new to it.

Thanks.

Here is the sample: alt text http://www.freeimagehosting.net/uploads/b03442fd36.png

alt text http://www.freeimagehosting.net/uploads/e6b36040e8.png

Due to the noise and many lines there, i am not able to determine which is the boundary line of the signboard. Sometimes I can find the boundary of the lines after hough transform. I am trapped....in this kind of scenario...

This is the 2 raw pictures taken from mobile phone camera

alt text http://www.freeimagehosting.net/uploads/6dbd613edf.jpg alt text http://www.freeimagehosting.net/uploads/720da20080.jpg

I need ya advice to see how i can process the image to get the signboard out?

Thank you very much

Answer 1

Both your images contain many lines that have nothing to do with the sign you're looking for. And some of those lines are longer/have higher contrast than the lines you actually want, so I think detecting the edge lines (e.g. using a hough transform or by summing up contrasts horizontally/vertically) will not work.

But: The sign you're looking for has other characteristics that should be easier to detect:

• There sign background has (almost) constant brightness
• It takes up a relatively large area of the image
• It's near the center of the image

So you're looking for a large connected area with low contrast. I've hacked a proof-of-concept algorithm in Mathematica. (I'm not an OpenCV expert, but I'll mention the respective OpenCV function when I know them.)

First, I use gaussian derivative filters to detect the gradient magnitude at each pixel. The gaussian derivative filter has a wide aperture (11x11 pixels in this case), so it is very noise-insensitive. I then normalize the gradient image to mean=1, so I can use the same thresholds for both samples.

src = Import["http://www.freeimagehosting.net/uploads/720da20080.jpg"];
pixels = ImageData[ColorConvert[src, "Grayscale"]];
gradient = Sqrt[GaussianFilter[pixels, 5, {1, 0}]^2 + GaussianFilter[pixels, 5, {0, 1}]^2];
gradient = gradient/Mean[Flatten[gradient]];

OpenCV implementation: You can use sepFilter2D for the actual filtering, but apparently, you'll have to calculate the filter kernel values yourself.

The result looks like this:

In this image, the sign background is dark and the sign borders are bright. So I can binarize this image and look for dark connected components.

binaryBorders = Binarize[Image[gradient], 0.2];
sign = DeleteBorderComponents@ColorNegate[binaryBorders];
largestComponent = SortBy[ComponentMeasurements[sign, {"Area", "ConvexVertices"}][[All, 2]], First][[-1, 2]];

OpenCV implementation: Thresholding should be straightforward, but I think OpenCV doesn't contain connected component analysis - you can either use flood fill or cvBlobsLib for that.

Now, just find the largest blob near the center of the image and find the convex hull (I've simply used the largest blob that's not connected to the background, but that might not be enough for every image).

Results:

Answer 2

Another approach that you can take that is more robust to noises in this situation is to generate a curve of the mean gray-level of the image along the x-axis and along the y-axis. That is, compute the mean gray level for each line/column in the image.

If, for example, the signboard (or its borders) is lighter than its surroundings (which is the case for all the examples shown in the question) you will have two peaks in your x-axis curve (for the left and right and borders) and two peaks in you y-axis curve (for the top and bottom borders). Using a border detection technique for a 1-dimension signal (maybe a high-pass filter) you can deduce the corners coordinates of the signboard.

I have seen this approach being used to to detect license plates and also for face recognition (the nose tends to be the brighter part of the face, so it generates a peak in the x-axis and y-axis curves).

Answer 3

This might be a little upside-down idea, but might be worth trying out. Instead of trying to detect the rectange and thinking of the text like noise, maybe you can treat the text like information and use it to detect the rectange easier.

Here's the outline of the idea:

• detect the text in the image. It does not need to be a robust implementation (you should be able to google some small library that does that, or maybe OpenCV has it), just a rough estimate of the text in the image
• find the cener of all the text detections. Even with noisy images it should be around the real place
• do Hough rectangle decetion on the area around the text detections. Maybe use the median distance from the center of the text detections, or something similar. Do different diameters and take the strongest recall.

The explanations and advantages:

• the area around the text is homogenous usually -- the real rectangle should be the first strong recall
• this way you do not have to do hough transform on the whole image, so you can do it more thurally (maybe multiple times on the same area but with different bucket sizes...)