Segmentation of half transparent material, e.g. glass

Question

I'm totally stuck on an issue regarding the segmentation of glassy objects. I need to get the object as precise as possible. My approaches were different. At first I tried to remove the background, so that only some sharp contours are left. But that only works for objects which have sharp edges / gradients. Otherwise the object itself is also removed. I posted two different images.

I tried to remove the background via morphological operations, like grayscale dilatation and a divison on it. but it didnt help much. after it, I tried a k-means with k=3 for getting the modified background separated from the gray and black values of the glass. That wasn't successfull in some cases, but not overall/in average. I also tried to make a canny edge detection with an overall blured filter, but that lead to weaker results in form of open contours, a lot of noise, etc. pp.

Canny with automatic threshold results:

testimg = imread('https://i.stack.imgur.com/Y6YO0.png');  
imshow(testimg)
imedges = edge(testimg,'canny');
imshow(imedges);

Same goes for the second image.

canny output #1 canny output #2

As you can see, there is a lot of noise inside and outside and doubled edges from the glas border. Even there are gaps in the edges.

So, I need your advices for getting a general approach for dealing with this problem of half-transparent materials, not for just these two images.

1) Other ideas for removing the background without damaging the object?

2) Other segmentation methods for getting the object separated from the background?

If it's possible, then with Matlab, IPT or statistical toolbox hints. Any other hints are also welcome!

Thank you for your answer in advance. Sincerely

Well subtracting the background from every image would be a start, making it more uniform: http://imgur.com/9WhcB — endolith, Dec 05 '12 at 01:14
@endolith Thanks for your effort, but I think that your posted example is not very good for further processings as you can see the 'ghost marks' of gradients/edges/differences. — mchlfchr, Dec 05 '12 at 02:04
What do you mean? Do you have a picture of the background without any glass? — endolith, Dec 05 '12 at 02:56
@endolith Nope, only the pictures with half-transparent objects inside. If I had pictures without objects I would have removed the background via subtraction. — mchlfchr, Dec 05 '12 at 09:29
Have you tried tracing the edges? If you can track the edge it should be easy to determine which part must remain. — Dennis Jaheruddin, Dec 05 '12 at 12:53
@DennisJaheruddin how would you recommend doing this? I am asking because I am afraid that there are no fully connected edges aka gaps in the edges. Regardless of the non-ideal sorroundings, e.g. noise, which are distracting the edges. Your further ideas are very welcome on that! — mchlfchr, Dec 05 '12 at 13:30
@mchlfchr: Do you have lots of images, with every point of the background showing up unobstructed in at least one of them? :) — endolith, Dec 05 '12 at 15:05
@endolith I have round about 10 images, but I have to process them independent of each other. Or what is your reason behind your question? — mchlfchr, Dec 05 '12 at 17:36
You should realize that an edge is not neccesarily a black line, I have good hopes that this answer can help you: http://stackoverflow.com/questions/10015857/find-edge-in-image — Dennis Jaheruddin, Dec 06 '12 at 10:26
@DennisJaheruddin I know that an edge is NOT a black line. An edge is defined as a change on intensity/frequency, what means that it's gray values changes more or less rapid. Nevertheless as you may see out of the context, the Canny method won't be the weapon of choice here, because of the background I will get a lot of noise (with Canny). And I can't predict the automatic threshold/sigma. So I need a method which elimates the background, but not the object itself. — mchlfchr, Dec 06 '12 at 12:34
I see all the potential hickups, but i still recommend you to run the command edge(I,'canny') and see what it can produce. I would have done it myself already if i could use that toolbox. Note that the canny setting already tries to filter out the noise. — Dennis Jaheruddin, Dec 06 '12 at 13:00
You could try using the berkeley edge detector followed by using edge connecting algorithms. — Naresh, Dec 07 '12 at 02:05
This problem is solved much easier by modifying your imaging setup. Are you in a position to do that ? — nav, Dec 10 '12 at 06:12
@nav: Nope, but I managed to find a solution. More or less precise. I will post it in the next few days, because I am very busy at the moment. Stay tuned. — mchlfchr, Dec 10 '12 at 16:30
OpenCV is trying some implementation of glass detection, where they have reached some high accuracy implementations, please check this answer for links : http://stackoverflow.com/questions/10168686/algorithm-improvement-for-coca-cola-can-shape-recognition/10219338#10219338 — Abid Rahman K, Dec 18 '12 at 16:11

score 3 · Accepted Answer · edited Dec 27 '12 at 09:37

Why not just use a simple 2D FFT (guassian) high pass filer?

I did this real quick using MATLAB

Shard #1 using high pass FFT:

The same thing is done on #2.

Shard #2 using high pass FFT:

As you can see, the background and glass area is wiped out, and only the edges are traced. I did not spend any time on it, but you can threshold the HP filtered output to have more crisp edges, or push the HP cut off higher.

Is this more the results you are looking to get?

score 1 · Answer 2 · edited Apr 13 '17 at 12:47

This is no attempt to answer the whole question, but I do have an idea about "cleaning the image".

You said you tried morphological operations already, and this is a variation to the idea, hopefully an upgrade.

This article: A. Vichik, R. Keshet, D. Malah: Self-dual morphology on tree semilattices and applications proposes a way to enhance on the classical morphological operators in a way that can add more desirable properties to them.

The article suggests to choose a hierarchical representation of an image according to desirable properties, and then proposes a method to define operators such as erosion, dilation, opening, top-hat on that representation. In their own words:

We have presented a general framework for producing new morphological operators (...)

I explained these hierarchical, tree-shaped structures in the second part of this answer (Semantic approaches), to which you can add Extrema-Watershed Tree mentioned in the article I linked here (and again).

It is an upgrade to (quoting the authors) "traditional grayscale mathematical morphology" because the operations keep the desirable properties of the representations. E.g. if your hierarchical representation is self-dual, your operators will be really self-dual (e.g. compare with quasi-self-dual opening-closing by reconstruction which is not really self-dual.)

The linked article also presents some results in filtering out the noise - you can compare their results from the article (and from the Thesis referenced in the article) to what you need (at least visually) and see if it would work for you before starting to code.

So, while choosing the simplest representation (max-/min-) tree will yield exactly the classical operations, choosing a self-dual tree which best suits your needs might give you a robust enough approach.

Segmentation of half transparent material, e.g. glass

2 Answers2